-- Logs begin at Tue 2022-10-11 11:20:06 UTC, end at Tue 2022-10-11 12:26:29 UTC. -- Oct 11 11:45:17.518109 dv-cinder-820994-6-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Oct 11 11:45:17.527799 dv-cinder-820994-6-iscsi devstack@keystone.service[81419]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Oct 11 11:45:17.530465 dv-cinder-820994-6-iscsi devstack@keystone.service[81419]: *** Starting uWSGI 2.0.18-debian (64bit) on [Tue Oct 11 11:45:17 2022] *** Oct 11 11:45:17.530465 dv-cinder-820994-6-iscsi devstack@keystone.service[81419]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Oct 11 11:45:17.530465 dv-cinder-820994-6-iscsi devstack@keystone.service[81419]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Oct 11 11:45:17.530465 dv-cinder-820994-6-iscsi devstack@keystone.service[81419]: nodename: dv-cinder-820994-6-iscsi Oct 11 11:45:17.530465 dv-cinder-820994-6-iscsi devstack@keystone.service[81419]: machine: x86_64 Oct 11 11:45:17.530465 dv-cinder-820994-6-iscsi devstack@keystone.service[81419]: clock source: unix Oct 11 11:45:17.530465 dv-cinder-820994-6-iscsi devstack@keystone.service[81419]: pcre jit disabled Oct 11 11:45:17.530465 dv-cinder-820994-6-iscsi devstack@keystone.service[81419]: detected number of CPU cores: 4 Oct 11 11:45:17.530465 dv-cinder-820994-6-iscsi devstack@keystone.service[81419]: current working directory: / Oct 11 11:45:17.530465 dv-cinder-820994-6-iscsi devstack@keystone.service[81419]: detected binary path: /usr/bin/uwsgi-core Oct 11 11:45:17.530465 dv-cinder-820994-6-iscsi devstack@keystone.service[81419]: your processes number limit is 47680 Oct 11 11:45:17.530465 dv-cinder-820994-6-iscsi devstack@keystone.service[81419]: your memory page size is 4096 bytes Oct 11 11:45:17.530465 dv-cinder-820994-6-iscsi devstack@keystone.service[81419]: detected max file descriptor number: 2048 Oct 11 11:45:17.530465 dv-cinder-820994-6-iscsi devstack@keystone.service[81419]: lock engine: pthread robust mutexes Oct 11 11:45:17.534092 dv-cinder-820994-6-iscsi devstack@keystone.service[81419]: thunder lock: enabled Oct 11 11:45:17.534092 dv-cinder-820994-6-iscsi devstack@keystone.service[81419]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Oct 11 11:45:17.534092 dv-cinder-820994-6-iscsi devstack@keystone.service[81419]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Oct 11 11:45:17.574727 dv-cinder-820994-6-iscsi devstack@keystone.service[81419]: Python main interpreter initialized at 0x563eacb899c0 Oct 11 11:45:17.574727 dv-cinder-820994-6-iscsi devstack@keystone.service[81419]: python threads support enabled Oct 11 11:45:17.574727 dv-cinder-820994-6-iscsi devstack@keystone.service[81419]: your server socket listen backlog is limited to 100 connections Oct 11 11:45:17.574727 dv-cinder-820994-6-iscsi devstack@keystone.service[81419]: your mercy for graceful operations on workers is 90 seconds Oct 11 11:45:17.575144 dv-cinder-820994-6-iscsi devstack@keystone.service[81419]: mapped 403077 bytes (393 KB) for 2 cores Oct 11 11:45:17.575144 dv-cinder-820994-6-iscsi devstack@keystone.service[81419]: *** Operational MODE: preforking *** Oct 11 11:45:17.575443 dv-cinder-820994-6-iscsi devstack@keystone.service[81419]: *** uWSGI is running in multiple interpreter mode *** Oct 11 11:45:17.575443 dv-cinder-820994-6-iscsi devstack@keystone.service[81419]: spawned uWSGI master process (pid: 81419) Oct 11 11:45:17.575675 dv-cinder-820994-6-iscsi systemd[1]: Started Devstack devstack@keystone.service. Oct 11 11:45:17.577842 dv-cinder-820994-6-iscsi devstack@keystone.service[81419]: spawned uWSGI worker 1 (pid: 81432, cores: 1) Oct 11 11:45:17.577842 dv-cinder-820994-6-iscsi devstack@keystone.service[81419]: spawned uWSGI worker 2 (pid: 81433, cores: 1) Oct 11 11:45:17.577842 dv-cinder-820994-6-iscsi devstack@keystone.service[81419]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Oct 11 11:45:21.832121 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 11 11:45:21.832121 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: from cryptography.utils import int_from_bytes Oct 11 11:45:21.833005 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 11 11:45:21.833005 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: from cryptography.utils import int_from_bytes Oct 11 11:45:21.833417 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 11 11:45:21.833417 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: from cryptography.utils import int_from_bytes Oct 11 11:45:21.833523 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 11 11:45:21.833523 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: from cryptography.utils import int_from_bytes Oct 11 11:45:21.881900 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 11 11:45:21.882113 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 11 11:45:21.882277 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] command line args: [] {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 11 11:45:21.882424 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 11 11:45:21.882515 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 11 11:45:21.882626 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 11 11:45:21.882703 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] ================================================================================ {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 11 11:45:21.882876 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] command line args: [] {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 11 11:45:21.882996 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] admin_token = **** {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 11:45:21.883104 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 11 11:45:21.883193 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] config_dir = [] {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 11:45:21.883272 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] ================================================================================ {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 11 11:45:21.883351 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 11:45:21.883506 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] config_source = [] {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 11:45:21.883651 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] admin_token = **** {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 11:45:21.883727 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] debug = True {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 11:45:21.883789 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] config_dir = [] {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 11:45:21.883906 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 11:45:21.884170 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 11:45:21.884170 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] config_source = [] {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 11:45:21.884285 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 11:45:21.884347 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] debug = True {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 11:45:21.884404 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 11:45:21.884467 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 11:45:21.884652 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 11:45:21.884735 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 11:45:21.884880 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 11:45:21.884967 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 11:45:21.885143 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] list_limit = None {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 11:45:21.885263 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 11:45:21.885336 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] log_config_append = None {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 11:45:21.885400 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 11:45:21.885458 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 11:45:21.885586 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] list_limit = None {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 11:45:21.885673 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] log_dir = None {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 11:45:21.885772 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] log_config_append = None {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 11:45:21.885870 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] log_file = None {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 11:45:21.885962 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 11:45:21.886060 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 11:45:21.886183 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] log_dir = None {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 11:45:21.886308 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 11:45:21.886425 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] log_file = None {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 11:45:21.886498 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 11:45:21.886585 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 11:45:21.886706 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 11:45:21.886825 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 11:45:21.886944 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 11:45:21.887026 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 11:45:21.887084 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 11:45:21.887257 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 11:45:21.887378 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 11:45:21.887519 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 11:45:21.887614 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 11:45:21.887768 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 11:45:21.887881 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 11:45:21.887980 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 11:45:21.888066 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 11:45:21.888180 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 11:45:21.888289 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 11:45:21.888395 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 11:45:21.888474 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 11:45:21.888583 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 11:45:21.888696 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 11:45:21.888779 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 11:45:21.888890 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 11:45:21.889002 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 11:45:21.889075 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 11:45:21.889192 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 11:45:21.889422 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 11:45:21.889553 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.59/identity {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 11:45:21.889708 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 11:45:21.889839 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] publish_errors = False {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 11:45:21.889917 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 11:45:21.890179 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 11:45:21.890315 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.59/identity {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 11:45:21.890396 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 11:45:21.890475 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] publish_errors = False {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 11:45:21.890648 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 11:45:21.890767 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 11:45:21.890847 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 11:45:21.890915 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 11:45:21.891001 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 11:45:21.891079 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] standard_threads = False {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 11:45:21.891234 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 11:45:21.891355 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 11:45:21.891430 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 11:45:21.891565 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 11:45:21.891655 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 11:45:21.891775 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] standard_threads = False {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 11:45:21.891850 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] use_journal = False {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 11:45:21.891918 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 11:45:21.891992 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] use_json = False {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 11:45:21.892130 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] use_stderr = False {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 11:45:21.892308 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 11:45:21.892389 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] use_syslog = False {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 11:45:21.892460 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 11:45:21.892460 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] use_journal = False {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 11:45:21.892590 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 11:45:21.892663 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] use_json = False {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 11:45:21.892740 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.892811 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] use_stderr = False {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 11:45:21.892935 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] use_syslog = False {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 11:45:21.893025 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.893187 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 11 11:45:21.893317 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.893406 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.893478 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.893656 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.893736 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.893905 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.894031 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.894208 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.894289 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.894442 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.894528 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.894710 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.894803 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.894961 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.895200 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.895278 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.895449 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.895534 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.895695 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.895785 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.895944 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.896029 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.896172 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.896258 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.896416 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.896504 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.896651 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.896735 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.896859 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.896931 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.897124 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.897215 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.897365 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.897442 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.897604 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.897688 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.897838 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.897911 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.898073 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.898152 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.898321 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.898405 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.898580 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.898659 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.898824 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.898910 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.899053 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.899128 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.899285 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.899362 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.899517 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.899603 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.899749 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.899829 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.899979 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.900055 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.900217 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.900292 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.900441 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.900513 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.900667 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.900750 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.900909 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.900984 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.901141 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.901215 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.901373 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.901447 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.901614 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.901688 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.901842 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.901922 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.902071 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.902147 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.902301 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.902376 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.902528 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.902650 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.902777 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.902854 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.903004 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.903077 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.903229 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.903349 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.903499 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.903570 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.903680 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.903757 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.903908 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.903982 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.904141 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.904217 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.904367 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.904443 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.904599 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.904673 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.904830 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.904906 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.905068 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.905142 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.905295 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.905367 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.905675 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.905796 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.905892 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.905984 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.906075 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.906167 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.906267 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.906360 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.906451 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.906611 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.906700 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.906823 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.906898 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.907009 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.907087 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.907159 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.907325 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.907420 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.907557 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.907691 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.907810 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.907894 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.908041 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.908133 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.908267 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.908359 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.908499 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.908956 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.909098 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.909187 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.909333 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.909417 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.909606 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.909696 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.909848 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.909923 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.910091 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.910180 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.910323 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.910408 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.910569 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.910663 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.910819 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.910950 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.911043 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.911103 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.911273 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.911416 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.911536 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.911662 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.911782 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.911909 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.911985 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.912051 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.912235 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.912366 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.912483 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.912557 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.912626 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.912694 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.912837 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.912927 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.913070 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.913143 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.913302 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.913373 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.913557 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.913641 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.913792 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.913866 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.914032 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.914107 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.914268 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.914356 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.914499 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.914601 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.914761 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.914836 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.914993 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.915148 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.915222 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.915290 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.915445 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.915520 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.915670 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.915754 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.915896 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.915972 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.916122 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.916197 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.916352 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.916428 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.916578 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.916658 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.916814 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.916888 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.917043 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.917122 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.917286 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.917362 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.917518 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.917592 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.917768 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.917856 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.918011 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.918088 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.918247 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.918322 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.918476 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] auth.external = None {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.918604 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.918741 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.918819 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.918995 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] auth.external = None {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.919081 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.919229 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.919305 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.919472 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] auth.password = None {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.919547 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.919702 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] auth.token = None {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.919794 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.919940 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.920062 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] auth.password = None {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.920179 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.920290 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] auth.token = None {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.920401 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.920503 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.920696 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.920802 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.920896 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.921000 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.921103 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.921202 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.921324 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.921431 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.921557 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] credential.caching = True {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.921659 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.921812 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.921912 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.922026 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] credential.caching = True {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.922127 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.922266 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.922365 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.922506 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.922659 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.922763 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.922858 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.922991 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.923092 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.923235 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.923344 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.923470 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.923576 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.923706 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.923812 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.923958 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.924060 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.924196 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.924301 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.924453 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.924553 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.924716 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.924818 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.924957 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.925070 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.925262 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.925370 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.925500 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] federation.caching = True {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.925600 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.925718 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.925845 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.925960 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.926036 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] federation.caching = True {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.926201 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.926278 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.926433 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.926508 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.926703 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.926801 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.926940 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.927019 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.927188 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.927263 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.927421 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.927504 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.927670 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.927750 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.927904 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.927981 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.928149 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.928233 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.928383 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] identity.caching = True {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.928459 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.928627 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.928703 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.928865 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] identity.caching = True {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.929017 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.929092 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.929167 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.929314 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.929391 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.929555 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.929639 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.929774 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.929869 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.930004 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.930086 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.930246 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.930335 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.930488 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.930627 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.930741 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.930817 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.930970 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.931058 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.931199 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.931276 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.931437 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.931513 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.931679 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.931789 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.931919 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.932001 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.932166 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.932243 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.932403 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.932490 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.932661 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.932736 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.932896 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.932976 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.933132 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.933276 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.933418 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.933510 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.933584 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.933654 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.933800 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.933889 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.934033 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.934110 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.934273 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.934350 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.934509 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.934617 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.934780 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.934856 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.935008 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.935083 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.935252 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.935336 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.935490 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.935588 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.935717 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.935794 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.935962 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.936040 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.936198 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.936274 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.936423 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.936499 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.936682 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.936781 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.936887 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.937014 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.937121 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.937209 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.937355 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.937433 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.937591 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.937666 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.937824 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.937900 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.938069 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.938183 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.938309 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.938435 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.938697 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.938778 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.938932 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.939009 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.939163 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.939238 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.939411 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.939487 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.939646 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.939723 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.939879 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.939954 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.940114 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.940197 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.940349 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.940427 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.940578 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] ldap.user = None {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.940661 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.940839 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.940919 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.941073 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] ldap.user = None {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.941151 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.941313 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.941388 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.941558 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.941634 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.941803 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.941880 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.942045 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.942120 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.942277 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.942353 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.942504 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.942596 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.942774 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.942850 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.943007 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.943082 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.943239 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.943316 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.943467 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.943544 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.943703 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.943782 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.943939 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.944017 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.944180 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.944255 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.944423 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.944499 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.944665 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.944743 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.944893 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.944969 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.945137 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.945212 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.945368 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.945445 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.945600 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.945674 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.945834 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.945910 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.946066 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.946150 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.946303 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.946378 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.946542 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.946716 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.946803 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.946874 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.947040 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.947116 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.947269 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.947363 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.947512 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.947589 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.947744 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.947826 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.947977 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.948053 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.948212 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.948289 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.948447 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.948522 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.948683 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.948757 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.948920 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.949005 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.949150 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.949227 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.949385 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.949460 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] resource.caching = True {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.949625 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.949703 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.949868 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] resource.caching = True {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.949945 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.950120 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.950370 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.950453 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.950633 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.950713 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.950887 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.950963 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.951140 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.951219 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.951379 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.951455 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.951632 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.951708 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.951867 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.951945 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.952100 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] role.caching = True {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.952183 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.952336 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] role.caching = True {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.952412 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] role.driver = sql {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.952565 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.952642 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] role.driver = sql {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.952807 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.952882 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.953058 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.953135 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.953295 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.953374 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.953541 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.953618 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.953789 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.953874 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.954030 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.954109 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.954274 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.954350 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.954517 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.954651 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.954773 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.954852 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.955019 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.955094 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.955262 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.955337 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.955506 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.955585 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.955749 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.955826 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.955996 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.956153 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.956426 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.956510 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.956692 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.956769 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.956941 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.957016 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.957189 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.957266 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.957434 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.957510 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.957667 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.957750 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.957911 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.957987 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.958524 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.958669 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.958767 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.958876 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.958973 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.959067 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.959160 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.959253 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.959345 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.959452 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.959548 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.959675 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.959773 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.959867 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.959960 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.960093 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.960194 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.960287 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.960381 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.960474 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.960567 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.960668 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.960763 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] token.caching = True {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.960856 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.960949 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.961042 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] token.caching = True {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.961134 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.961234 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.961330 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.961423 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.961575 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.961677 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.961825 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.961935 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.962056 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.962157 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.962300 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.962401 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.963036 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.963115 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.963175 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.963233 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.963289 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.963345 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.963409 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.963471 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.963542 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.963600 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.963702 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.963857 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.963972 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.964053 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.964204 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.964290 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.964445 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.964524 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.964708 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.964794 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.964968 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] database.connection = **** {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.965059 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.965209 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.965286 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] database.connection = **** {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.965454 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.965548 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.965690 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.965776 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.965929 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.966007 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.966162 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.966239 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.966397 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.966482 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.966656 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.966740 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.966896 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.967005 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.967118 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.967250 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.967371 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.967483 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.967597 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.967703 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.967812 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.967954 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.968077 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.968198 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.968304 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.968527 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.968651 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.968770 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.968891 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.968998 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.969120 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.969227 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.969349 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.969471 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.969595 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81432) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 11 11:45:21.969718 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 11 11:45:21.969718 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81433) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 11 11:45:21.999269 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81432) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 11:45:21.999485 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81433) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 11:45:21.999561 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81432) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 11:45:21.999639 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81433) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 11:45:21.999712 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81432) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 11:45:21.999787 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81433) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 11:45:22.008118 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81432) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 11:45:22.008260 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81433) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 11:45:22.036300 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81432) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 11:45:22.036498 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81432) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 11:45:22.036834 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81433) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 11:45:22.037083 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81433) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 11:45:22.089180 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81432) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 11:45:22.089495 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81433) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 11:45:22.137596 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81432) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 11:45:22.137792 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81432) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 11:45:22.138280 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81433) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 11:45:22.138533 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81433) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 11:45:22.297716 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81432) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 11:45:22.297923 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81432) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 11:45:22.298144 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81432) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 11:45:22.298380 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81432) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 11:45:22.298543 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81432) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 11:45:22.298639 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81433) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 11:45:22.298809 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81432) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 11:45:22.298932 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81433) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 11:45:22.299093 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81433) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 11:45:22.299320 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81433) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 11:45:22.299544 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81433) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 11:45:22.299764 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81433) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 11:45:22.322095 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81432) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 11:45:22.322318 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81432) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 11:45:22.322520 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81432) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 11:45:22.322659 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81433) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 11:45:22.322767 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81432) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 11:45:22.322851 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81433) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 11:45:22.323018 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81432) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 11:45:22.323108 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81433) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 11:45:22.323213 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81432) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 11:45:22.323313 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81433) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 11:45:22.323429 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81432) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 11:45:22.323561 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81433) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 11:45:22.323840 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81433) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 11:45:22.324036 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81433) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 11:45:22.334686 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81432) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 11:45:22.335306 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81433) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 11:45:22.347448 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81432) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 11:45:22.347589 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81432) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 11:45:22.347828 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81432) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 11:45:22.347971 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81433) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 11:45:22.348056 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81432) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 11:45:22.348136 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81433) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 11:45:22.348206 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81432) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 11:45:22.348362 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81433) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 11:45:22.348504 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81432) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 11:45:22.348577 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81433) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 11:45:22.348653 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81432) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 11:45:22.348760 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81433) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 11:45:22.349006 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81433) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 11:45:22.349225 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81433) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 11 11:45:22.365331 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.365519 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 11 11:45:22.365882 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.366131 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 11 11:45:22.366254 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.366344 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.366425 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 11 11:45:22.366584 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.366710 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.366884 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.367029 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 11 11:45:22.367102 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.367179 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.367462 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.367731 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.367966 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.376685 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.376969 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.377234 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.377522 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.377646 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.377921 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.378198 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.378539 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.384555 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81432) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 11 11:45:22.385836 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81433) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 11 11:45:22.387465 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81432) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 11 11:45:22.387731 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.388032 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.388318 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.388575 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.388751 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81433) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 11 11:45:22.388915 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.389051 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.389143 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.389359 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.389484 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.389714 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.389845 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.389970 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.390094 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.390217 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.390355 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.390499 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.390735 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.391011 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.391292 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.391547 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.413468 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.414959 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81432) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 11 11:45:22.415083 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.415223 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.416567 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81433) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 11 11:45:22.416863 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.419877 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81432) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 11 11:45:22.420162 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.420451 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.421566 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81433) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 11 11:45:22.421875 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.422207 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.426797 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81432) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 11 11:45:22.427087 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.428802 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81433) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 11 11:45:22.429076 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.432808 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81432) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 11 11:45:22.433102 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.433391 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.433682 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.433965 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.434242 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.434484 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.434812 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.434936 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81433) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 11 11:45:22.435210 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.435504 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.435806 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.436070 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.436349 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.436626 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.436911 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.450272 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.450548 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.452849 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.453136 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.454397 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81432) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 11 11:45:22.457065 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81433) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 11 11:45:22.457670 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.457970 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.460322 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.460593 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.462182 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81432) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 11 11:45:22.464861 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81433) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 11 11:45:22.465286 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81432) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 11 11:45:22.468319 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81433) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 11 11:45:22.468446 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.468745 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.469038 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.469325 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.469603 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.469886 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.471761 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.472285 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.472926 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.473391 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.473831 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.474134 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.485874 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81432) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 11 11:45:22.486162 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.486435 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.486732 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.490297 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81433) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 11 11:45:22.490603 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.490903 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.491178 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.493466 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.494993 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.496472 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.496749 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.498090 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.499439 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81432) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 11 11:45:22.499666 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.499756 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.500000 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.500282 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.500671 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.501207 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.501476 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.504270 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81433) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 11 11:45:22.504579 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.504850 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.505136 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.505584 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.512114 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81432) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 11 11:45:22.512406 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.512697 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.512948 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.513353 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.513643 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.513924 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.517035 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81433) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 11 11:45:22.517325 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.517611 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.517900 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.518288 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.518603 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.518871 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.529228 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81432) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 11 11:45:22.532143 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81432) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 11 11:45:22.535129 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81433) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 11 11:45:22.535541 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.537000 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.538056 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81433) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 11 11:45:22.538452 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81432) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 11 11:45:22.538757 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.539171 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.540985 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.542477 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.543982 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81433) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 11 11:45:22.544271 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.544683 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.545463 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.546947 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81432) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 11 11:45:22.549809 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.550225 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.550513 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.550820 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.551363 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.552837 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81433) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 11 11:45:22.555775 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.556175 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.556473 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.556762 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.558595 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81432) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 11 11:45:22.558879 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.559160 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.564642 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81433) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 11 11:45:22.564938 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.565228 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.566040 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81432) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 11 11:45:22.566323 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.566596 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.566916 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.567188 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.567458 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.567733 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.568010 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.568282 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.568543 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.568829 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.569089 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.569372 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.569635 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81432) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.572301 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81433) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 11 11:45:22.572577 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.572904 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.573208 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.573506 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.573822 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.574120 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.574433 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.574763 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.575076 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.575365 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.575637 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.575912 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.576317 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81433) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 11 11:45:22.604055 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x563eacb899c0 pid: 81432 (default app) Oct 11 11:45:22.607423 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc72b42b-3bff-49e0-89c1-0edd02687dd0 None None] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:45:22.607709 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc72b42b-3bff-49e0-89c1-0edd02687dd0 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:45:22.607976 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc72b42b-3bff-49e0-89c1-0edd02687dd0 None None] PATH_INFO: `/v3` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:45:22.610817 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x563eacb899c0 pid: 81433 (default app) Oct 11 11:45:22.612116 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1/1] 10.222.0.59 () {50 vars in 791 bytes} [Tue Oct 11 11:45:22 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:45:29.470899 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b769b2a2-cfc3-48da-8585-44899d8c660a None None] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:45:29.471441 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b769b2a2-cfc3-48da-8585-44899d8c660a None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:45:29.471441 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b769b2a2-cfc3-48da-8585-44899d8c660a None None] PATH_INFO: `/` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:45:29.475982 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1/2] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:45:29 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 11 11:45:29.481515 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbdd975-5840-4f1b-897d-40f54dc696a6 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:45:29.481978 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbdd975-5840-4f1b-897d-40f54dc696a6 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:45:29.482117 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbdd975-5840-4f1b-897d-40f54dc696a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:45:29.552604 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: INFO dbcounter [None req-6dbdd975-5840-4f1b-897d-40f54dc696a6 None None] Registered counter for database keystone Oct 11 11:45:29.571904 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG oslo_db.sqlalchemy.engines [None req-6dbdd975-5840-4f1b-897d-40f54dc696a6 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81432) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 11 11:45:29.579394 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writer thread running {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 11 11:45:29.757225 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG passlib.handlers.bcrypt [None req-6dbdd975-5840-4f1b-897d-40f54dc696a6 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81432) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 11 11:45:29.757506 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG passlib.handlers.bcrypt [None req-6dbdd975-5840-4f1b-897d-40f54dc696a6 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81432) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 11 11:45:29.816454 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-6dbdd975-5840-4f1b-897d-40f54dc696a6 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:45:29.818071 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-6dbdd975-5840-4f1b-897d-40f54dc696a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:45:29.948172 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 2/3] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:45:29 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 468 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 11 11:45:30.410148 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b769b2a2-cfc3-48da-8585-44899d8c660a None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:45:30.481007 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: INFO dbcounter [None req-b769b2a2-cfc3-48da-8585-44899d8c660a None None] Registered counter for database keystone Oct 11 11:45:30.503422 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG oslo_db.sqlalchemy.engines [None req-b769b2a2-cfc3-48da-8585-44899d8c660a None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81433) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 11 11:45:30.510831 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writer thread running {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 11 11:45:30.629254 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b769b2a2-cfc3-48da-8585-44899d8c660a None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:45:30.633972 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8780e9-818d-4b2e-8b3c-496c9db83146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:45:30.634635 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8780e9-818d-4b2e-8b3c-496c9db83146 admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:45:30.634915 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8780e9-818d-4b2e-8b3c-496c9db83146 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:45:30.635186 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8780e9-818d-4b2e-8b3c-496c9db83146 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:45:30.643746 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8780e9-818d-4b2e-8b3c-496c9db83146 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:45:30.696143 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: WARNING keystone.common.rbac_enforcer.enforcer [None req-2a8780e9-818d-4b2e-8b3c-496c9db83146 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 11 11:45:30.696397 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8780e9-818d-4b2e-8b3c-496c9db83146 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:45:30.706739 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 2/4] 10.222.0.59 () {58 vars in 1324 bytes} [Tue Oct 11 11:45:30 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 11 11:45:30.713240 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbdd975-5840-4f1b-897d-40f54dc696a6 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:45:30.732035 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbdd975-5840-4f1b-897d-40f54dc696a6 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:45:30.737723 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3b1e7d-0630-47a3-8496-96ef7abe1e75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:45:30.738935 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3b1e7d-0630-47a3-8496-96ef7abe1e75 admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:45:30.739219 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3b1e7d-0630-47a3-8496-96ef7abe1e75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:45:30.739466 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3b1e7d-0630-47a3-8496-96ef7abe1e75 admin admin] PATH_INFO: `/v3/projects/ec9b5254352a410fb1f7cca5a5393294` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:45:30.741227 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3b1e7d-0630-47a3-8496-96ef7abe1e75 admin admin] RBAC: Authorizing `identity:get_project(project_id=ec9b5254352a410fb1f7cca5a5393294)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:45:30.794418 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: WARNING keystone.common.rbac_enforcer.enforcer [None req-ed3b1e7d-0630-47a3-8496-96ef7abe1e75 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 11 11:45:30.794788 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3b1e7d-0630-47a3-8496-96ef7abe1e75 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:45:30.799468 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 3/5] 10.222.0.59 () {56 vars in 1207 bytes} [Tue Oct 11 11:45:30 2022] GET /identity/v3/projects/ec9b5254352a410fb1f7cca5a5393294 => generated 344 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:45:32.924130 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69531d77-0be4-4e8b-9076-eb3e5af2de38 None None] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:45:32.924729 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69531d77-0be4-4e8b-9076-eb3e5af2de38 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:45:32.924729 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69531d77-0be4-4e8b-9076-eb3e5af2de38 None None] PATH_INFO: `/` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:45:32.928818 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 3/6] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:45:32 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:45:32.934794 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60c2b90-aa0a-4cf0-8f75-d1b8ea15afd6 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:45:32.935254 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60c2b90-aa0a-4cf0-8f75-d1b8ea15afd6 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:45:32.935438 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60c2b90-aa0a-4cf0-8f75-d1b8ea15afd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:45:32.957047 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-a60c2b90-aa0a-4cf0-8f75-d1b8ea15afd6 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:45:32.958574 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-a60c2b90-aa0a-4cf0-8f75-d1b8ea15afd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:45:32.968329 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 4/7] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:45:32 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 11 11:45:33.236213 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69531d77-0be4-4e8b-9076-eb3e5af2de38 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:45:33.250181 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69531d77-0be4-4e8b-9076-eb3e5af2de38 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:45:33.255168 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c85d65-070f-4ea2-87dc-1197668ceba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:45:33.255780 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c85d65-070f-4ea2-87dc-1197668ceba6 admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:45:33.256062 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c85d65-070f-4ea2-87dc-1197668ceba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:45:33.256343 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c85d65-070f-4ea2-87dc-1197668ceba6 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:45:33.264543 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c85d65-070f-4ea2-87dc-1197668ceba6 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:45:33.266334 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c85d65-070f-4ea2-87dc-1197668ceba6 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:45:33.275646 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 4/8] 10.222.0.59 () {58 vars in 1324 bytes} [Tue Oct 11 11:45:33 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 11 11:45:33.281826 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60c2b90-aa0a-4cf0-8f75-d1b8ea15afd6 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:45:33.291864 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60c2b90-aa0a-4cf0-8f75-d1b8ea15afd6 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:45:33.297551 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39def69a-5d25-460b-8950-fafe00b94b58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:45:33.298189 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39def69a-5d25-460b-8950-fafe00b94b58 admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:45:33.298467 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39def69a-5d25-460b-8950-fafe00b94b58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:45:33.298765 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39def69a-5d25-460b-8950-fafe00b94b58 admin admin] PATH_INFO: `/v3/users/a1613582a9844c5fa354d54653ab8e01` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:45:33.300220 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39def69a-5d25-460b-8950-fafe00b94b58 admin admin] RBAC: Authorizing `identity:get_user(user_id=a1613582a9844c5fa354d54653ab8e01)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:45:33.302154 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39def69a-5d25-460b-8950-fafe00b94b58 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:45:33.303759 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 5/9] 10.222.0.59 () {56 vars in 1198 bytes} [Tue Oct 11 11:45:33 2022] GET /identity/v3/users/a1613582a9844c5fa354d54653ab8e01 => generated 247 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:45:35.481954 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c407a337-7276-43dd-a054-01563b731f23 None None] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:45:35.482527 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c407a337-7276-43dd-a054-01563b731f23 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:45:35.482527 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c407a337-7276-43dd-a054-01563b731f23 None None] PATH_INFO: `/` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:45:35.483877 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 5/10] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:45:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:45:35.489469 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414d8f4b-d6ba-4535-971b-b92f51ed7dce None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:45:35.489910 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414d8f4b-d6ba-4535-971b-b92f51ed7dce None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:45:35.489999 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414d8f4b-d6ba-4535-971b-b92f51ed7dce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:45:35.509370 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-414d8f4b-d6ba-4535-971b-b92f51ed7dce None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:45:35.510587 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-414d8f4b-d6ba-4535-971b-b92f51ed7dce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:45:35.545637 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101ca15e-d7e2-4719-abbd-ed954c6f772d None None] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:45:35.545898 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101ca15e-d7e2-4719-abbd-ed954c6f772d None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:45:35.546165 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101ca15e-d7e2-4719-abbd-ed954c6f772d None None] PATH_INFO: `/` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:45:35.547532 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 6/11] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:45:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:45:35.552984 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0e4509-0d35-45db-8f5f-96a802a7c00b None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:45:35.553265 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0e4509-0d35-45db-8f5f-96a802a7c00b None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:45:35.553534 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0e4509-0d35-45db-8f5f-96a802a7c00b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:45:35.594054 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 6/12] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:45:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 11:45:35.596079 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG passlib.handlers.bcrypt [None req-9d0e4509-0d35-45db-8f5f-96a802a7c00b None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81433) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 11 11:45:35.596482 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG passlib.handlers.bcrypt [None req-9d0e4509-0d35-45db-8f5f-96a802a7c00b None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81433) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 11 11:45:35.645335 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-9d0e4509-0d35-45db-8f5f-96a802a7c00b None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:45:35.647002 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-9d0e4509-0d35-45db-8f5f-96a802a7c00b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:45:35.751201 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 7/13] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:45:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 11:45:35.854879 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414d8f4b-d6ba-4535-971b-b92f51ed7dce None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:45:35.882445 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:45:35.968988 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414d8f4b-d6ba-4535-971b-b92f51ed7dce None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:45:36.008895 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53b4445-57a0-4177-9180-9c930fd4c0af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:45:36.009435 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53b4445-57a0-4177-9180-9c930fd4c0af None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:45:36.009746 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53b4445-57a0-4177-9180-9c930fd4c0af None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:45:36.010019 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53b4445-57a0-4177-9180-9c930fd4c0af None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:45:36.012196 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0e4509-0d35-45db-8f5f-96a802a7c00b None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:45:36.022843 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53b4445-57a0-4177-9180-9c930fd4c0af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 11:45:36.031928 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application [None req-a53b4445-57a0-4177-9180-9c930fd4c0af None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 11 11:45:36.031928 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:45:36.031928 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:45:36.031928 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:45:36.031928 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:45:36.031928 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:45:36.031928 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:45:36.031928 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:45:36.031928 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:45:36.031928 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:45:36.031928 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:45:36.031928 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:45:36.031928 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 11:45:36.031928 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 11:45:36.031928 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 11:45:36.031928 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 11:45:36.032877 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 11:45:36.032877 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 11:45:36.032877 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 11:45:36.032877 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 11:45:36.032877 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:45:36.032877 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:45:36.032877 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:45:36.032877 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:45:36.032877 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:45:36.032877 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:45:36.032877 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:45:36.032877 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application with Lock( Oct 11 11:45:36.032877 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:45:36.032877 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:45:36.032877 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:45:36.032877 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:45:36.034343 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:45:36.034343 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:45:36.034343 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:45:36.034343 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:45:36.034343 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 11:45:36.034343 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 11:45:36.034343 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 11:45:36.034343 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 11:45:36.034343 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 11:45:36.034343 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 11:45:36.034343 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 11 11:45:36.034343 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:45:36.034343 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 7/14] 10.222.0.59 () {56 vars in 1096 bytes} [Tue Oct 11 11:45:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:45:36.039529 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53b4445-57a0-4177-9180-9c930fd4c0af None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:45:36.115352 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0e4509-0d35-45db-8f5f-96a802a7c00b None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:45:36.117719 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:45:36.124623 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:45:36.144785 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53b4445-57a0-4177-9180-9c930fd4c0af None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:45:36.153753 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16570d8e-8bb4-4f5f-9f5f-cdcf695fa270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:45:36.154660 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16570d8e-8bb4-4f5f-9f5f-cdcf695fa270 None admin] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:45:36.154938 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16570d8e-8bb4-4f5f-9f5f-cdcf695fa270 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:45:36.155207 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16570d8e-8bb4-4f5f-9f5f-cdcf695fa270 None admin] PATH_INFO: `/v3/domains` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:45:36.156158 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16570d8e-8bb4-4f5f-9f5f-cdcf695fa270 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:45:36.157594 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16570d8e-8bb4-4f5f-9f5f-cdcf695fa270 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:45:36.170011 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.sql.core [None req-16570d8e-8bb4-4f5f-9f5f-cdcf695fa270 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 11 11:45:36.170011 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 11 11:45:36.170011 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [parameters: {'id': '2084c0647db5402dbd28f263d6ca3d7f', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 11 11:45:36.170011 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81433) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 11 11:45:36.180182 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7f9d0d-d376-4553-b5fc-72910a13edc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:45:36.180809 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7f9d0d-d376-4553-b5fc-72910a13edc8 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:45:36.181088 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7f9d0d-d376-4553-b5fc-72910a13edc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:45:36.181367 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7f9d0d-d376-4553-b5fc-72910a13edc8 None admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:45:36.182076 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7f9d0d-d376-4553-b5fc-72910a13edc8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:45:36.182380 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7f9d0d-d376-4553-b5fc-72910a13edc8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:45:36.183844 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7f9d0d-d376-4553-b5fc-72910a13edc8 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:45:36.190069 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application [None req-16570d8e-8bb4-4f5f-9f5f-cdcf695fa270 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 11 11:45:36.190069 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:45:36.190069 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 11 11:45:36.190069 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 11 11:45:36.190069 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 11 11:45:36.190069 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 11 11:45:36.190069 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 11 11:45:36.190069 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application result = self._query(query) Oct 11 11:45:36.190069 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 11 11:45:36.190069 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application conn.query(q) Oct 11 11:45:36.190069 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 11 11:45:36.190069 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 11 11:45:36.190069 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 11 11:45:36.190069 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application result.read() Oct 11 11:45:36.191107 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 11 11:45:36.191107 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 11 11:45:36.191107 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 11 11:45:36.191107 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 11 11:45:36.191107 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 11 11:45:36.191107 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 11 11:45:36.191107 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 11 11:45:36.191107 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 11 11:45:36.191107 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 11 11:45:36.191107 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:45:36.191107 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 11 11:45:36.191107 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:45:36.191107 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:45:36.191107 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 11 11:45:36.191107 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 11 11:45:36.191107 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 11 11:45:36.191107 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 11 11:45:36.191107 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 11 11:45:36.192033 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application next(self.gen) Oct 11 11:45:36.192033 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 11 11:45:36.192033 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application yield resource Oct 11 11:45:36.192033 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 11 11:45:36.192033 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application next(self.gen) Oct 11 11:45:36.192033 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 11 11:45:36.192033 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application self.session.rollback() Oct 11 11:45:36.192033 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 11 11:45:36.192033 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application self.force_reraise() Oct 11 11:45:36.192033 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 11 11:45:36.192033 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application raise self.value Oct 11 11:45:36.192033 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 11 11:45:36.192033 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 11 11:45:36.192033 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 11 11:45:36.192033 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application session.commit() Oct 11 11:45:36.192033 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 11 11:45:36.192033 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 11 11:45:36.192875 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 11 11:45:36.192875 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application self._prepare_impl() Oct 11 11:45:36.192875 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 11 11:45:36.192875 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application self.session.flush() Oct 11 11:45:36.192875 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 11 11:45:36.192875 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application self._flush(objects) Oct 11 11:45:36.192875 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 11 11:45:36.192875 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 11 11:45:36.192875 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 11 11:45:36.192875 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application compat.raise_( Oct 11 11:45:36.192875 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 11 11:45:36.192875 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application raise exception Oct 11 11:45:36.192875 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 11 11:45:36.192875 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application flush_context.execute() Oct 11 11:45:36.192875 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 11 11:45:36.192875 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application rec.execute(self) Oct 11 11:45:36.192875 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 11 11:45:36.193732 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 11 11:45:36.193732 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 11 11:45:36.193732 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 11 11:45:36.193732 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 11 11:45:36.193732 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 11 11:45:36.193732 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 11 11:45:36.193732 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 11 11:45:36.193732 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 11 11:45:36.193732 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 11 11:45:36.193732 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 11 11:45:36.193732 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 11 11:45:36.193732 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 11 11:45:36.193732 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 11 11:45:36.193732 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 11 11:45:36.193732 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 11 11:45:36.194454 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 11 11:45:36.194454 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application raise exception Oct 11 11:45:36.194454 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 11 11:45:36.194454 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 11 11:45:36.194454 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 11 11:45:36.194454 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 11 11:45:36.194454 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 11 11:45:36.194454 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application result = self._query(query) Oct 11 11:45:36.194454 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 11 11:45:36.194454 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application conn.query(q) Oct 11 11:45:36.194454 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 11 11:45:36.194454 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 11 11:45:36.194454 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 11 11:45:36.194454 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application result.read() Oct 11 11:45:36.194454 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 11 11:45:36.194454 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 11 11:45:36.194454 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 11 11:45:36.195304 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 11 11:45:36.195304 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 11 11:45:36.195304 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 11 11:45:36.195304 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 11 11:45:36.195304 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 11 11:45:36.195304 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 11 11:45:36.195304 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 11 11:45:36.195304 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application [parameters: {'id': '2084c0647db5402dbd28f263d6ca3d7f', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 11 11:45:36.195304 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 11 11:45:36.195304 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:45:36.195304 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 11 11:45:36.195304 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:45:36.195304 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:45:36.195304 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 11 11:45:36.195304 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 11 11:45:36.196040 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 11 11:45:36.196040 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 11 11:45:36.196040 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name Default. Oct 11 11:45:36.196040 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:45:36.196040 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 11 11:45:36.196040 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:45:36.196040 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:45:36.196040 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:45:36.196040 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:45:36.196040 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:45:36.196040 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:45:36.196040 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:45:36.196040 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:45:36.196040 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:45:36.196040 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:45:36.196040 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:45:36.196040 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:45:36.196931 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 128, in post Oct 11 11:45:36.196931 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_domain( Oct 11 11:45:36.196931 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:45:36.196931 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:45:36.196931 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 765, in create_domain Oct 11 11:45:36.196931 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application is_domain_project = self.create_project( Oct 11 11:45:36.196931 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:45:36.196931 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:45:36.196931 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 11 11:45:36.196931 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 11 11:45:36.196931 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 11 11:45:36.196931 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:45:36.196931 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 8/15] 10.222.0.59 () {60 vars in 1137 bytes} [Tue Oct 11 11:45:36 2022] POST /identity/v3/domains => generated 190 bytes in 180 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Oct 11 11:45:36.197557 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 8/16] 10.222.0.59 () {56 vars in 1099 bytes} [Tue Oct 11 11:45:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 11:45:36.199260 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16570d8e-8bb4-4f5f-9f5f-cdcf695fa270 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:45:36.202162 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7f9d0d-d376-4553-b5fc-72910a13edc8 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:45:36.303745 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16570d8e-8bb4-4f5f-9f5f-cdcf695fa270 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:45:36.308922 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7f9d0d-d376-4553-b5fc-72910a13edc8 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:45:36.341790 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a689db9-d967-4eec-88de-b05f1f8f71bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:45:36.342745 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a689db9-d967-4eec-88de-b05f1f8f71bd None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:45:36.343023 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a689db9-d967-4eec-88de-b05f1f8f71bd None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:45:36.343290 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a689db9-d967-4eec-88de-b05f1f8f71bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:45:36.344933 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ea6a02-acd9-410c-951f-f3172e03aa95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:45:36.345586 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ea6a02-acd9-410c-951f-f3172e03aa95 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:45:36.345870 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ea6a02-acd9-410c-951f-f3172e03aa95 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:45:36.346140 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ea6a02-acd9-410c-951f-f3172e03aa95 None admin] PATH_INFO: `/v3/users/a1613582a9844c5fa354d54653ab8e01` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:45:36.347332 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ea6a02-acd9-410c-951f-f3172e03aa95 None admin] RBAC: Authorizing `identity:get_user(user_id=a1613582a9844c5fa354d54653ab8e01)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:45:36.348815 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ea6a02-acd9-410c-951f-f3172e03aa95 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:45:36.350375 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 9/17] 10.222.0.59 () {56 vars in 1177 bytes} [Tue Oct 11 11:45:36 2022] GET /identity/v3/users/a1613582a9844c5fa354d54653ab8e01 => generated 247 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 11:45:36.356016 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ea6a02-acd9-410c-951f-f3172e03aa95 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:45:36.362468 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a689db9-d967-4eec-88de-b05f1f8f71bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:45:36.364097 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a689db9-d967-4eec-88de-b05f1f8f71bd None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:45:36.371588 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:45:36.381065 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application [None req-4a689db9-d967-4eec-88de-b05f1f8f71bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 11:45:36.381065 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:45:36.381065 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:45:36.381065 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:45:36.381065 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:45:36.381065 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:45:36.381065 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:45:36.381065 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:45:36.381065 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:45:36.381065 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:45:36.381065 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:45:36.381065 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:45:36.381065 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 11:45:36.381065 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 11:45:36.381065 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 11:45:36.381065 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 11:45:36.382110 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:45:36.382110 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:45:36.382110 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:45:36.382110 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:45:36.382110 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:45:36.382110 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:45:36.382110 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:45:36.382110 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application with Lock( Oct 11 11:45:36.382110 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:45:36.382110 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:45:36.382110 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:45:36.382110 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:45:36.382110 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:45:36.382110 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:45:36.382110 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:45:36.382110 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:45:36.382110 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 11:45:36.382961 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 11:45:36.382961 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 11:45:36.382961 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:45:36.382961 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 9/18] 10.222.0.59 () {56 vars in 1108 bytes} [Tue Oct 11 11:45:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:45:36.388402 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a689db9-d967-4eec-88de-b05f1f8f71bd None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:45:36.412168 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:45:36.465359 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ea6a02-acd9-410c-951f-f3172e03aa95 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:45:36.496737 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a689db9-d967-4eec-88de-b05f1f8f71bd None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:45:36.503266 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cac53f5-14e3-4bc4-891e-04e3aa8cdbc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:45:36.504004 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cac53f5-14e3-4bc4-891e-04e3aa8cdbc3 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:45:36.504281 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cac53f5-14e3-4bc4-891e-04e3aa8cdbc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:45:36.504565 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cac53f5-14e3-4bc4-891e-04e3aa8cdbc3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:45:36.505991 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cac53f5-14e3-4bc4-891e-04e3aa8cdbc3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:45:36.507608 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cac53f5-14e3-4bc4-891e-04e3aa8cdbc3 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:45:36.509392 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 10/19] 10.222.0.59 () {56 vars in 1108 bytes} [Tue Oct 11 11:45:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 11:45:36.515794 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cac53f5-14e3-4bc4-891e-04e3aa8cdbc3 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:45:36.540890 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba00de16-c677-4e8c-8f47-d60fe56ca7b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:45:36.541572 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba00de16-c677-4e8c-8f47-d60fe56ca7b1 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:45:36.541855 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba00de16-c677-4e8c-8f47-d60fe56ca7b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:45:36.542125 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba00de16-c677-4e8c-8f47-d60fe56ca7b1 None admin] PATH_INFO: `/v3/domains` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:45:36.542868 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba00de16-c677-4e8c-8f47-d60fe56ca7b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:45:36.543165 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba00de16-c677-4e8c-8f47-d60fe56ca7b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:45:36.544633 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba00de16-c677-4e8c-8f47-d60fe56ca7b1 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:45:36.564964 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 10/20] 10.222.0.59 () {56 vars in 1109 bytes} [Tue Oct 11 11:45:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 11:45:36.609597 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:45:36.633311 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cac53f5-14e3-4bc4-891e-04e3aa8cdbc3 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:45:36.672462 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6863084e-3bcf-41a5-ad84-f03a28fad17b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:45:36.673157 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6863084e-3bcf-41a5-ad84-f03a28fad17b None admin] REQUEST_METHOD: `PUT` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:45:36.673437 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6863084e-3bcf-41a5-ad84-f03a28fad17b None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:45:36.673708 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6863084e-3bcf-41a5-ad84-f03a28fad17b None admin] PATH_INFO: `/v3/domains/default/users/a1613582a9844c5fa354d54653ab8e01/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:45:36.676095 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6863084e-3bcf-41a5-ad84-f03a28fad17b None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=a1613582a9844c5fa354d54653ab8e01, role_id=5cf1f4bd2fef46ab9c6eff56d52f9f69)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:45:36.677585 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6863084e-3bcf-41a5-ad84-f03a28fad17b None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:45:36.795882 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 11/21] 10.222.0.59 () {58 vars in 1361 bytes} [Tue Oct 11 11:45:36 2022] PUT /identity/v3/domains/default/users/a1613582a9844c5fa354d54653ab8e01/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 11:45:39.876543 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dc71ec-c070-439b-8077-6a2ca16fae43 None None] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:45:39.877237 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dc71ec-c070-439b-8077-6a2ca16fae43 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:45:39.877237 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dc71ec-c070-439b-8077-6a2ca16fae43 None None] PATH_INFO: `/` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:45:39.878766 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 11/22] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:45:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:45:39.888193 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8868d0b-1357-436f-a0a1-1412a5a403b4 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:45:39.889509 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8868d0b-1357-436f-a0a1-1412a5a403b4 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:45:39.890212 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8868d0b-1357-436f-a0a1-1412a5a403b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:45:39.941982 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-c8868d0b-1357-436f-a0a1-1412a5a403b4 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:45:39.945364 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-c8868d0b-1357-436f-a0a1-1412a5a403b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:45:39.965652 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71763bd-1012-4d3c-a4f0-11c32f8dd66f None None] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:45:39.965920 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71763bd-1012-4d3c-a4f0-11c32f8dd66f None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:45:39.966220 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71763bd-1012-4d3c-a4f0-11c32f8dd66f None None] PATH_INFO: `/` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:45:39.967908 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 12/23] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:45:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:45:39.978799 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dcd8f0-d6c2-4916-979a-cef362d56a7a None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:45:39.978799 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dcd8f0-d6c2-4916-979a-cef362d56a7a None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:45:39.978799 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dcd8f0-d6c2-4916-979a-cef362d56a7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:45:40.023118 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-55dcd8f0-d6c2-4916-979a-cef362d56a7a None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:45:40.024767 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-55dcd8f0-d6c2-4916-979a-cef362d56a7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:45:40.128179 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 12/24] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:45:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 11 11:45:40.130196 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32deaff6-243d-4905-917a-6ee4ae4ed104 None None] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:45:40.130649 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32deaff6-243d-4905-917a-6ee4ae4ed104 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:45:40.131064 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32deaff6-243d-4905-917a-6ee4ae4ed104 None None] PATH_INFO: `/` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:45:40.137233 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 13/25] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:45:40 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:45:40.138691 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b73de10-783c-49b9-a3d4-0cbc94f8f90f None None] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:45:40.139115 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b73de10-783c-49b9-a3d4-0cbc94f8f90f None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:45:40.139526 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b73de10-783c-49b9-a3d4-0cbc94f8f90f None None] PATH_INFO: `/` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:45:40.141176 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 14/26] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:45:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:45:40.146898 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc0d1aa-e95f-4e33-85fa-7f44f27e765b None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:45:40.147182 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc0d1aa-e95f-4e33-85fa-7f44f27e765b None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:45:40.147460 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc0d1aa-e95f-4e33-85fa-7f44f27e765b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:45:40.166432 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-afc0d1aa-e95f-4e33-85fa-7f44f27e765b None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:45:40.167696 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-afc0d1aa-e95f-4e33-85fa-7f44f27e765b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:45:40.201154 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 13/27] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:45:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 11:45:40.206143 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf200a8-d166-4d35-b19f-0748e0bd7703 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:45:40.206490 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf200a8-d166-4d35-b19f-0748e0bd7703 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:45:40.206853 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf200a8-d166-4d35-b19f-0748e0bd7703 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:45:40.238889 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-ebf200a8-d166-4d35-b19f-0748e0bd7703 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:45:40.245650 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-ebf200a8-d166-4d35-b19f-0748e0bd7703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:45:40.256726 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:45:40.286621 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:45:40.293144 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 15/28] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:45:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 11:45:40.417196 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 14/29] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:45:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 11:45:40.721600 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc0d1aa-e95f-4e33-85fa-7f44f27e765b None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:45:40.731179 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf200a8-d166-4d35-b19f-0748e0bd7703 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:45:40.847423 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc0d1aa-e95f-4e33-85fa-7f44f27e765b None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:45:40.884978 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49d2663-fe43-4063-b103-023622e4fb13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:45:40.885611 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49d2663-fe43-4063-b103-023622e4fb13 None admin] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:45:40.885933 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49d2663-fe43-4063-b103-023622e4fb13 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:45:40.886388 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49d2663-fe43-4063-b103-023622e4fb13 None admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:45:40.887166 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49d2663-fe43-4063-b103-023622e4fb13 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:45:40.889392 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49d2663-fe43-4063-b103-023622e4fb13 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:45:40.912596 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 16/30] 10.222.0.59 () {60 vars in 1131 bytes} [Tue Oct 11 11:45:40 2022] POST /identity/v3/roles => generated 225 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 11:45:40.929978 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49d2663-fe43-4063-b103-023622e4fb13 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:45:41.012325 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf200a8-d166-4d35-b19f-0748e0bd7703 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:45:41.055727 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4d16c9-ceae-4f4a-8eac-4fdcff4c07cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:45:41.056384 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4d16c9-ceae-4f4a-8eac-4fdcff4c07cc None admin] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:45:41.056666 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4d16c9-ceae-4f4a-8eac-4fdcff4c07cc None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:45:41.056941 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4d16c9-ceae-4f4a-8eac-4fdcff4c07cc None admin] PATH_INFO: `/v3/roles` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:45:41.057680 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4d16c9-ceae-4f4a-8eac-4fdcff4c07cc None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:45:41.059211 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4d16c9-ceae-4f4a-8eac-4fdcff4c07cc None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:45:41.091740 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:45:41.137692 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:45:41.223240 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 15/31] 10.222.0.59 () {60 vars in 1131 bytes} [Tue Oct 11 11:45:40 2022] POST /identity/v3/roles => generated 219 bytes in 493 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 11:45:41.224629 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4d16c9-ceae-4f4a-8eac-4fdcff4c07cc None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:45:41.228235 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49d2663-fe43-4063-b103-023622e4fb13 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:45:41.372873 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c034b23d-1711-40a4-a7ee-d4ed5aa69dd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:45:41.373996 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c034b23d-1711-40a4-a7ee-d4ed5aa69dd6 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:45:41.374504 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c034b23d-1711-40a4-a7ee-d4ed5aa69dd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:45:41.375020 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c034b23d-1711-40a4-a7ee-d4ed5aa69dd6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:45:41.422438 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c034b23d-1711-40a4-a7ee-d4ed5aa69dd6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 11:45:41.443141 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application [None req-c034b23d-1711-40a4-a7ee-d4ed5aa69dd6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 11 11:45:41.443141 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:45:41.443141 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:45:41.443141 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:45:41.443141 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:45:41.443141 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:45:41.443141 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:45:41.443141 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:45:41.443141 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:45:41.443141 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:45:41.443141 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:45:41.443141 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:45:41.443141 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 11:45:41.443141 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 11:45:41.443141 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 11:45:41.443141 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 11:45:41.445157 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 11:45:41.445157 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 11:45:41.445157 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 11:45:41.445157 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 11:45:41.445157 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:45:41.445157 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:45:41.445157 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:45:41.445157 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:45:41.445157 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:45:41.445157 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:45:41.445157 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:45:41.445157 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application with Lock( Oct 11 11:45:41.445157 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:45:41.445157 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:45:41.445157 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:45:41.445157 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:45:41.446907 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:45:41.446907 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:45:41.446907 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:45:41.446907 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:45:41.446907 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 11:45:41.446907 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 11:45:41.446907 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 11:45:41.446907 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 11:45:41.446907 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 11:45:41.446907 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 11:45:41.446907 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 11 11:45:41.446907 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:45:41.451115 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 17/32] 10.222.0.59 () {56 vars in 1096 bytes} [Tue Oct 11 11:45:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 538 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:45:41.460146 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c034b23d-1711-40a4-a7ee-d4ed5aa69dd6 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:45:41.548047 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4d16c9-ceae-4f4a-8eac-4fdcff4c07cc None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:45:41.601130 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c034b23d-1711-40a4-a7ee-d4ed5aa69dd6 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:45:41.626473 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e79cd46-7522-4df3-8efd-7235e97f2a82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:45:41.626963 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:45:41.628325 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e79cd46-7522-4df3-8efd-7235e97f2a82 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:45:41.629143 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e79cd46-7522-4df3-8efd-7235e97f2a82 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:45:41.629840 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e79cd46-7522-4df3-8efd-7235e97f2a82 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:45:41.642808 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8340d1a1-59fd-4534-a1da-7e1fa3b5932c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:45:41.643441 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8340d1a1-59fd-4534-a1da-7e1fa3b5932c None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:45:41.643721 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8340d1a1-59fd-4534-a1da-7e1fa3b5932c None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:45:41.644108 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8340d1a1-59fd-4534-a1da-7e1fa3b5932c None admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:45:41.644791 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8340d1a1-59fd-4534-a1da-7e1fa3b5932c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:45:41.645095 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8340d1a1-59fd-4534-a1da-7e1fa3b5932c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:45:41.647261 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8340d1a1-59fd-4534-a1da-7e1fa3b5932c None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:45:41.652684 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e79cd46-7522-4df3-8efd-7235e97f2a82 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:45:41.654317 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e79cd46-7522-4df3-8efd-7235e97f2a82 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:45:41.662634 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 18/33] 10.222.0.59 () {56 vars in 1099 bytes} [Tue Oct 11 11:45:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:45:41.669211 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8340d1a1-59fd-4534-a1da-7e1fa3b5932c None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:45:41.675618 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application [None req-7e79cd46-7522-4df3-8efd-7235e97f2a82 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 11:45:41.675618 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:45:41.675618 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:45:41.675618 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:45:41.675618 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:45:41.675618 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:45:41.675618 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:45:41.675618 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:45:41.675618 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:45:41.675618 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:45:41.675618 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:45:41.675618 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:45:41.675618 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 11:45:41.675618 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 11:45:41.675618 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 11:45:41.675618 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 11:45:41.677251 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:45:41.677251 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:45:41.677251 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:45:41.677251 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:45:41.677251 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:45:41.677251 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:45:41.677251 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:45:41.677251 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application with Lock( Oct 11 11:45:41.677251 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:45:41.677251 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:45:41.677251 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:45:41.677251 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:45:41.677251 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:45:41.677251 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:45:41.677251 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:45:41.677251 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:45:41.677251 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 11:45:41.678812 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 11:45:41.678812 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 11:45:41.678812 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:45:41.682980 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 16/34] 10.222.0.59 () {56 vars in 1108 bytes} [Tue Oct 11 11:45:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 453 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:45:41.684391 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e79cd46-7522-4df3-8efd-7235e97f2a82 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:45:41.767135 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:45:41.817864 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8340d1a1-59fd-4534-a1da-7e1fa3b5932c None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:45:41.831013 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e79cd46-7522-4df3-8efd-7235e97f2a82 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:45:41.872777 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835ac7de-3d0f-489f-995f-e29641fbc4ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:45:41.873470 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835ac7de-3d0f-489f-995f-e29641fbc4ae None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:45:41.873754 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835ac7de-3d0f-489f-995f-e29641fbc4ae None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:45:41.874025 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835ac7de-3d0f-489f-995f-e29641fbc4ae None admin] PATH_INFO: `/v3/users/a1613582a9844c5fa354d54653ab8e01` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:45:41.875282 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835ac7de-3d0f-489f-995f-e29641fbc4ae None admin] RBAC: Authorizing `identity:get_user(user_id=a1613582a9844c5fa354d54653ab8e01)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:45:41.876870 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835ac7de-3d0f-489f-995f-e29641fbc4ae None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:45:41.878603 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 19/35] 10.222.0.59 () {56 vars in 1177 bytes} [Tue Oct 11 11:45:41 2022] GET /identity/v3/users/a1613582a9844c5fa354d54653ab8e01 => generated 247 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 11:45:41.879910 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc211c3b-4b1b-4c64-af4d-2e3f298b4534 None None] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:45:41.880208 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc211c3b-4b1b-4c64-af4d-2e3f298b4534 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:45:41.880465 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc211c3b-4b1b-4c64-af4d-2e3f298b4534 None None] PATH_INFO: `/` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:45:41.882025 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 20/36] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:45:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:45:41.887223 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc211c3b-4b1b-4c64-af4d-2e3f298b4534 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:45:41.891442 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58d54de-4313-44a2-9f52-12cb7b75fd31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:45:41.892266 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58d54de-4313-44a2-9f52-12cb7b75fd31 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:45:41.892681 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58d54de-4313-44a2-9f52-12cb7b75fd31 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:45:41.893365 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58d54de-4313-44a2-9f52-12cb7b75fd31 None admin] PATH_INFO: `/v3/domains` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:45:41.894376 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58d54de-4313-44a2-9f52-12cb7b75fd31 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:45:41.894845 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58d54de-4313-44a2-9f52-12cb7b75fd31 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:45:41.896424 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58d54de-4313-44a2-9f52-12cb7b75fd31 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:45:41.918610 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 17/37] 10.222.0.59 () {56 vars in 1109 bytes} [Tue Oct 11 11:45:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:45:41.918610 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350f1e8c-cb1a-4ff0-bc85-9bc0a7656436 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:45:41.918610 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350f1e8c-cb1a-4ff0-bc85-9bc0a7656436 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:45:41.918610 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350f1e8c-cb1a-4ff0-bc85-9bc0a7656436 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:45:41.947478 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-350f1e8c-cb1a-4ff0-bc85-9bc0a7656436 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:45:41.949643 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-350f1e8c-cb1a-4ff0-bc85-9bc0a7656436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:45:41.952804 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:45:42.033104 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 18/38] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:45:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 11:45:42.033998 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350f1e8c-cb1a-4ff0-bc85-9bc0a7656436 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:45:42.088807 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:45:42.142991 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc211c3b-4b1b-4c64-af4d-2e3f298b4534 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:45:42.165476 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350f1e8c-cb1a-4ff0-bc85-9bc0a7656436 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:45:42.196349 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fdbc50-c33f-40dd-a4ba-e8ac70fbe22a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:45:42.197867 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fdbc50-c33f-40dd-a4ba-e8ac70fbe22a None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:45:42.198323 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fdbc50-c33f-40dd-a4ba-e8ac70fbe22a None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:45:42.198815 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fdbc50-c33f-40dd-a4ba-e8ac70fbe22a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:45:42.200857 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fdbc50-c33f-40dd-a4ba-e8ac70fbe22a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:45:42.204299 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fdbc50-c33f-40dd-a4ba-e8ac70fbe22a None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:45:42.212154 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 21/39] 10.222.0.59 () {56 vars in 1108 bytes} [Tue Oct 11 11:45:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:45:42.212154 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f135ba-f4c8-4892-9610-867ae07e84aa None None] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:45:42.212154 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f135ba-f4c8-4892-9610-867ae07e84aa None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:45:42.212154 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f135ba-f4c8-4892-9610-867ae07e84aa None None] PATH_INFO: `/` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:45:42.212154 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 22/40] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:45:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:45:42.218143 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e190f8b-8e0e-44da-9ac3-6967d5515e18 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:45:42.218457 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e190f8b-8e0e-44da-9ac3-6967d5515e18 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:45:42.218825 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e190f8b-8e0e-44da-9ac3-6967d5515e18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:45:42.231376 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639fcdd3-b683-49ec-8599-98d05dd4dc9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:45:42.232178 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639fcdd3-b683-49ec-8599-98d05dd4dc9b None admin] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:45:42.232595 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639fcdd3-b683-49ec-8599-98d05dd4dc9b None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:45:42.233589 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639fcdd3-b683-49ec-8599-98d05dd4dc9b None admin] PATH_INFO: `/v3/projects` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:45:42.234683 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639fcdd3-b683-49ec-8599-98d05dd4dc9b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:45:42.236312 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639fcdd3-b683-49ec-8599-98d05dd4dc9b None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:45:42.239301 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-4e190f8b-8e0e-44da-9ac3-6967d5515e18 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:45:42.240544 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-4e190f8b-8e0e-44da-9ac3-6967d5515e18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:45:42.276330 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 19/41] 10.222.0.59 () {60 vars in 1141 bytes} [Tue Oct 11 11:45:42 2022] POST /identity/v3/projects => generated 301 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 11:45:42.277325 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639fcdd3-b683-49ec-8599-98d05dd4dc9b None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:45:42.370612 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 23/42] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:45:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 11 11:45:42.386604 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e190f8b-8e0e-44da-9ac3-6967d5515e18 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:45:42.469243 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:45:42.472750 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:45:42.492101 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639fcdd3-b683-49ec-8599-98d05dd4dc9b None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:45:42.543280 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d29771d-034f-4812-b340-52f97d1c00f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:45:42.544063 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d29771d-034f-4812-b340-52f97d1c00f2 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:45:42.544408 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d29771d-034f-4812-b340-52f97d1c00f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:45:42.544768 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d29771d-034f-4812-b340-52f97d1c00f2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:45:42.545618 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d29771d-034f-4812-b340-52f97d1c00f2 None admin] RBAC: Adding query filter params (role.id=5cf1f4bd2fef46ab9c6eff56d52f9f69, scope.domain.id=default, user.id=a1613582a9844c5fa354d54653ab8e01) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:45:42.545994 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d29771d-034f-4812-b340-52f97d1c00f2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:45:42.547641 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d29771d-034f-4812-b340-52f97d1c00f2 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:45:42.558236 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 20/43] 10.222.0.59 () {56 vars in 1322 bytes} [Tue Oct 11 11:45:42 2022] GET /identity/v3/role_assignments?user.id=a1613582a9844c5fa354d54653ab8e01&scope.domain.id=default&role.id=5cf1f4bd2fef46ab9c6eff56d52f9f69 => generated 530 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:45:42.570051 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e190f8b-8e0e-44da-9ac3-6967d5515e18 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:45:42.634052 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56996a1-3f90-4ba7-8d24-6f7ed2c15ee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:45:42.634977 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56996a1-3f90-4ba7-8d24-6f7ed2c15ee1 None admin] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:45:42.635401 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56996a1-3f90-4ba7-8d24-6f7ed2c15ee1 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:45:42.635804 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56996a1-3f90-4ba7-8d24-6f7ed2c15ee1 None admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:45:42.636626 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56996a1-3f90-4ba7-8d24-6f7ed2c15ee1 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:45:42.638270 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56996a1-3f90-4ba7-8d24-6f7ed2c15ee1 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:45:42.656932 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 24/44] 10.222.0.59 () {60 vars in 1131 bytes} [Tue Oct 11 11:45:42 2022] POST /identity/v3/roles => generated 223 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 11:45:42.671645 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d29771d-034f-4812-b340-52f97d1c00f2 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:45:42.822686 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d29771d-034f-4812-b340-52f97d1c00f2 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:45:42.873341 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1748371-06a4-4b2d-bacf-690dc7ac7915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:45:42.874111 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1748371-06a4-4b2d-bacf-690dc7ac7915 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:45:42.874542 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1748371-06a4-4b2d-bacf-690dc7ac7915 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:45:42.875020 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1748371-06a4-4b2d-bacf-690dc7ac7915 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:45:42.876618 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1748371-06a4-4b2d-bacf-690dc7ac7915 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:45:42.878851 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1748371-06a4-4b2d-bacf-690dc7ac7915 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:45:42.880746 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 21/45] 10.222.0.59 () {56 vars in 1108 bytes} [Tue Oct 11 11:45:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:45:42.886625 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56996a1-3f90-4ba7-8d24-6f7ed2c15ee1 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:45:43.002711 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56996a1-3f90-4ba7-8d24-6f7ed2c15ee1 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:45:43.013372 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:45:43.044521 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7aa934d-e6ba-482c-bd0f-c872646e2080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:45:43.045178 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7aa934d-e6ba-482c-bd0f-c872646e2080 None admin] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:45:43.045457 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7aa934d-e6ba-482c-bd0f-c872646e2080 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:45:43.045732 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7aa934d-e6ba-482c-bd0f-c872646e2080 None admin] PATH_INFO: `/v3/projects` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:45:43.046409 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7aa934d-e6ba-482c-bd0f-c872646e2080 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:45:43.047962 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7aa934d-e6ba-482c-bd0f-c872646e2080 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:45:43.083707 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 25/46] 10.222.0.59 () {60 vars in 1141 bytes} [Tue Oct 11 11:45:42 2022] POST /identity/v3/projects => generated 312 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 11:45:45.259261 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e5a795-a901-42d7-a6c2-fa760bc75e4d None None] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:45:45.259778 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e5a795-a901-42d7-a6c2-fa760bc75e4d None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:45:45.259778 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e5a795-a901-42d7-a6c2-fa760bc75e4d None None] PATH_INFO: `/` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:45:45.261084 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 22/47] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:45:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:45:45.266437 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef91608c-2def-49d0-be41-31db73a46e7f None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:45:45.266888 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef91608c-2def-49d0-be41-31db73a46e7f None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:45:45.267023 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef91608c-2def-49d0-be41-31db73a46e7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:45:45.287096 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-ef91608c-2def-49d0-be41-31db73a46e7f None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:45:45.288316 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-ef91608c-2def-49d0-be41-31db73a46e7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:45:45.358918 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 26/48] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:45:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 11:45:45.631250 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e5a795-a901-42d7-a6c2-fa760bc75e4d None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:45:45.660472 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:45:45.735218 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e5a795-a901-42d7-a6c2-fa760bc75e4d None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:45:45.769883 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915a414a-fe89-45af-96db-3573f6a8639a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:45:45.770517 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915a414a-fe89-45af-96db-3573f6a8639a None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:45:45.770835 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915a414a-fe89-45af-96db-3573f6a8639a None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:45:45.771110 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915a414a-fe89-45af-96db-3573f6a8639a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:45:45.772469 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915a414a-fe89-45af-96db-3573f6a8639a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:45:45.773977 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915a414a-fe89-45af-96db-3573f6a8639a None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:45:45.775610 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 23/49] 10.222.0.59 () {56 vars in 1108 bytes} [Tue Oct 11 11:45:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:45:45.781475 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef91608c-2def-49d0-be41-31db73a46e7f None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:45:45.873854 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:45:45.885970 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef91608c-2def-49d0-be41-31db73a46e7f None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:45:45.922182 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eeeea4f-f320-4a49-9e21-1dee9804a1ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:45:45.922841 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeeea4f-f320-4a49-9e21-1dee9804a1ca None admin] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:45:45.923120 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeeea4f-f320-4a49-9e21-1dee9804a1ca None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:45:45.923391 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeeea4f-f320-4a49-9e21-1dee9804a1ca None admin] PATH_INFO: `/v3/projects` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:45:45.924037 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eeeea4f-f320-4a49-9e21-1dee9804a1ca None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:45:45.925590 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eeeea4f-f320-4a49-9e21-1dee9804a1ca None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:45:45.954504 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 27/50] 10.222.0.59 () {60 vars in 1140 bytes} [Tue Oct 11 11:45:45 2022] POST /identity/v3/projects => generated 298 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 11:45:48.104899 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9394521c-5f7f-4c04-a676-ee70349db94c None None] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:45:48.105484 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9394521c-5f7f-4c04-a676-ee70349db94c None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:45:48.105484 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9394521c-5f7f-4c04-a676-ee70349db94c None None] PATH_INFO: `/` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:45:48.106840 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 24/51] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:45:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:45:48.112204 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366375e9-682c-44d6-b1ca-50b8ea44ded4 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:45:48.112632 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366375e9-682c-44d6-b1ca-50b8ea44ded4 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:45:48.112744 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366375e9-682c-44d6-b1ca-50b8ea44ded4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:45:48.132444 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-366375e9-682c-44d6-b1ca-50b8ea44ded4 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:45:48.133641 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-366375e9-682c-44d6-b1ca-50b8ea44ded4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:45:48.205893 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 28/52] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:45:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 11:45:48.467002 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9394521c-5f7f-4c04-a676-ee70349db94c None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:45:48.565101 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:45:48.573318 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9394521c-5f7f-4c04-a676-ee70349db94c None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:45:48.607815 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5329bf40-44ed-4620-98ef-7ae54941338e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:45:48.608441 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5329bf40-44ed-4620-98ef-7ae54941338e None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:45:48.608717 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5329bf40-44ed-4620-98ef-7ae54941338e None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:45:48.608988 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5329bf40-44ed-4620-98ef-7ae54941338e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:45:48.610153 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5329bf40-44ed-4620-98ef-7ae54941338e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:45:48.611622 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5329bf40-44ed-4620-98ef-7ae54941338e None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:45:48.613210 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 25/53] 10.222.0.59 () {56 vars in 1108 bytes} [Tue Oct 11 11:45:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:45:48.619146 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366375e9-682c-44d6-b1ca-50b8ea44ded4 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:45:48.691375 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:45:48.725523 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366375e9-682c-44d6-b1ca-50b8ea44ded4 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:45:48.761472 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50b9d1b-c2a2-4376-a881-addc89705a12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:45:48.762097 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50b9d1b-c2a2-4376-a881-addc89705a12 None admin] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:45:48.762377 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50b9d1b-c2a2-4376-a881-addc89705a12 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:45:48.762661 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50b9d1b-c2a2-4376-a881-addc89705a12 None admin] PATH_INFO: `/v3/users` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:45:48.763343 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50b9d1b-c2a2-4376-a881-addc89705a12 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:45:48.764796 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50b9d1b-c2a2-4376-a881-addc89705a12 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:45:48.792676 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 29/54] 10.222.0.59 () {60 vars in 1132 bytes} [Tue Oct 11 11:45:48 2022] POST /identity/v3/users => generated 275 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 11:45:52.358297 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f217f40-9464-4e30-92e5-b71711be8f1b None None] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:45:52.359318 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f217f40-9464-4e30-92e5-b71711be8f1b None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:45:52.359806 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f217f40-9464-4e30-92e5-b71711be8f1b None None] PATH_INFO: `/` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:45:52.361389 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 26/55] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:45:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:45:52.373778 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bdfdf5-d78b-4ee6-b8b6-c26254b82035 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:45:52.373778 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bdfdf5-d78b-4ee6-b8b6-c26254b82035 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:45:52.373778 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bdfdf5-d78b-4ee6-b8b6-c26254b82035 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:45:52.395446 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-e1bdfdf5-d78b-4ee6-b8b6-c26254b82035 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:45:52.396796 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-e1bdfdf5-d78b-4ee6-b8b6-c26254b82035 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:45:52.437492 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5c65fb-d8ab-4858-a42c-28bd5d7b3ca7 None None] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:45:52.438337 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5c65fb-d8ab-4858-a42c-28bd5d7b3ca7 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:45:52.439019 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5c65fb-d8ab-4858-a42c-28bd5d7b3ca7 None None] PATH_INFO: `/` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:45:52.442177 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 27/56] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:45:52 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:45:52.451239 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff6bd34-2865-4a88-bf64-fdf3f11f56d9 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:45:52.452137 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff6bd34-2865-4a88-bf64-fdf3f11f56d9 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:45:52.452877 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff6bd34-2865-4a88-bf64-fdf3f11f56d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:45:52.481597 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-dff6bd34-2865-4a88-bf64-fdf3f11f56d9 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:45:52.483089 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-dff6bd34-2865-4a88-bf64-fdf3f11f56d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:45:52.519255 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 30/57] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:45:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 11:45:52.588962 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 28/58] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:45:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 11 11:45:52.854054 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56f330e-bac0-46db-93c9-75259f52cf05 None None] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:45:52.854784 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56f330e-bac0-46db-93c9-75259f52cf05 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:45:52.855269 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56f330e-bac0-46db-93c9-75259f52cf05 None None] PATH_INFO: `/` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:45:52.856875 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 31/59] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:45:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:45:52.862702 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679c6a65-363b-4da6-9686-933d5703521f None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:45:52.863325 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679c6a65-363b-4da6-9686-933d5703521f None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:45:52.863938 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679c6a65-363b-4da6-9686-933d5703521f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:45:52.865042 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56f330e-bac0-46db-93c9-75259f52cf05 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:45:52.899970 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-679c6a65-363b-4da6-9686-933d5703521f None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:45:52.901224 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-679c6a65-363b-4da6-9686-933d5703521f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:45:52.947819 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:45:52.979136 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 29/60] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:45:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 11:45:52.986582 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fa3135-e15b-45c8-b3bb-1b0cd363cc26 None None] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:45:52.987585 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fa3135-e15b-45c8-b3bb-1b0cd363cc26 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:45:52.988122 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fa3135-e15b-45c8-b3bb-1b0cd363cc26 None None] PATH_INFO: `/` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:45:52.989786 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 30/61] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:45:52 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:45:52.996442 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207d52a8-4fad-4903-a128-ea1146a3489e None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:45:52.997039 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207d52a8-4fad-4903-a128-ea1146a3489e None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:45:52.997498 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207d52a8-4fad-4903-a128-ea1146a3489e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:45:53.009715 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56f330e-bac0-46db-93c9-75259f52cf05 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:45:53.032983 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-207d52a8-4fad-4903-a128-ea1146a3489e None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:45:53.035088 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-207d52a8-4fad-4903-a128-ea1146a3489e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:45:53.050841 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7690e7-c3f7-4bc6-a00a-8a69102b24ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:45:53.051808 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7690e7-c3f7-4bc6-a00a-8a69102b24ae None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:45:53.052243 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7690e7-c3f7-4bc6-a00a-8a69102b24ae None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:45:53.052657 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7690e7-c3f7-4bc6-a00a-8a69102b24ae None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:45:53.065574 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7690e7-c3f7-4bc6-a00a-8a69102b24ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 11:45:53.070905 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:45:53.071938 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application [None req-dc7690e7-c3f7-4bc6-a00a-8a69102b24ae None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 11 11:45:53.071938 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:45:53.071938 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:45:53.071938 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:45:53.071938 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:45:53.071938 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:45:53.071938 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:45:53.071938 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:45:53.071938 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:45:53.071938 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:45:53.071938 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:45:53.071938 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:45:53.071938 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 11:45:53.071938 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 11:45:53.071938 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 11:45:53.071938 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 11:45:53.072770 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 11:45:53.072770 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 11:45:53.072770 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 11:45:53.072770 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 11:45:53.072770 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:45:53.072770 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:45:53.072770 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:45:53.072770 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:45:53.072770 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:45:53.072770 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:45:53.072770 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:45:53.072770 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application with Lock( Oct 11 11:45:53.072770 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:45:53.072770 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:45:53.072770 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:45:53.072770 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:45:53.073554 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:45:53.073554 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:45:53.073554 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:45:53.073554 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:45:53.073554 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 11:45:53.073554 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 11:45:53.073554 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 11:45:53.073554 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 11:45:53.073554 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 11:45:53.073554 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 11:45:53.073554 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 11 11:45:53.073554 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:45:53.076273 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 32/62] 10.222.0.59 () {56 vars in 1096 bytes} [Tue Oct 11 11:45:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:45:53.082478 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7690e7-c3f7-4bc6-a00a-8a69102b24ae None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:45:53.168716 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 31/63] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:45:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Oct 11 11:45:53.169601 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207d52a8-4fad-4903-a128-ea1146a3489e None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:45:53.232611 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7690e7-c3f7-4bc6-a00a-8a69102b24ae None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:45:53.269251 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:45:53.303844 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad208ee-3cbe-41ee-878f-7c7242c5b87c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:45:53.305443 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad208ee-3cbe-41ee-878f-7c7242c5b87c None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:45:53.306174 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad208ee-3cbe-41ee-878f-7c7242c5b87c None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:45:53.306897 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad208ee-3cbe-41ee-878f-7c7242c5b87c None admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:45:53.308265 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad208ee-3cbe-41ee-878f-7c7242c5b87c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:45:53.309083 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad208ee-3cbe-41ee-878f-7c7242c5b87c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:45:53.312000 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad208ee-3cbe-41ee-878f-7c7242c5b87c None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:45:53.329338 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 33/64] 10.222.0.59 () {56 vars in 1099 bytes} [Tue Oct 11 11:45:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:45:53.333572 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e415680-dc97-48fd-bfb3-761faf1cba9e None None] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:45:53.335015 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e415680-dc97-48fd-bfb3-761faf1cba9e None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:45:53.336142 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e415680-dc97-48fd-bfb3-761faf1cba9e None None] PATH_INFO: `/` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:45:53.340673 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 34/65] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:45:53 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:45:53.342363 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e415680-dc97-48fd-bfb3-761faf1cba9e None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:45:53.426740 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207d52a8-4fad-4903-a128-ea1146a3489e None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:45:53.489168 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e415680-dc97-48fd-bfb3-761faf1cba9e None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:45:53.493629 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1a19a5-5d0e-4265-8105-a78e5c8ec1d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:45:53.494282 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1a19a5-5d0e-4265-8105-a78e5c8ec1d9 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:45:53.494584 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1a19a5-5d0e-4265-8105-a78e5c8ec1d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:45:53.494872 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1a19a5-5d0e-4265-8105-a78e5c8ec1d9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:45:53.507774 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1a19a5-5d0e-4265-8105-a78e5c8ec1d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 11:45:53.514315 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application [None req-8c1a19a5-5d0e-4265-8105-a78e5c8ec1d9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 11 11:45:53.514315 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:45:53.514315 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:45:53.514315 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:45:53.514315 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:45:53.514315 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:45:53.514315 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:45:53.514315 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:45:53.514315 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:45:53.514315 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:45:53.514315 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:45:53.514315 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:45:53.514315 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 11:45:53.514315 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 11:45:53.514315 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 11:45:53.514315 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 11:45:53.515333 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 11:45:53.515333 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 11:45:53.515333 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 11:45:53.515333 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 11:45:53.515333 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:45:53.515333 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:45:53.515333 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:45:53.515333 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:45:53.515333 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:45:53.515333 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:45:53.515333 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:45:53.515333 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application with Lock( Oct 11 11:45:53.515333 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:45:53.515333 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:45:53.515333 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:45:53.515333 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:45:53.516074 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:45:53.516074 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:45:53.516074 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:45:53.516074 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:45:53.516074 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 11:45:53.516074 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 11:45:53.516074 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 11:45:53.516074 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 11:45:53.516074 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 11:45:53.516074 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 11:45:53.516074 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 11 11:45:53.516074 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:45:53.516074 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 32/66] 10.222.0.59 () {56 vars in 1099 bytes} [Tue Oct 11 11:45:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 346 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:45:53.516773 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7155ba-d2d0-48b2-9cd5-407dceef7063 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:45:53.517041 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7155ba-d2d0-48b2-9cd5-407dceef7063 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:45:53.517327 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7155ba-d2d0-48b2-9cd5-407dceef7063 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:45:53.532663 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e37bd91-f896-41fc-ba25-4480ff576fae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:45:53.533364 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e37bd91-f896-41fc-ba25-4480ff576fae None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:45:53.533831 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e37bd91-f896-41fc-ba25-4480ff576fae None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:45:53.534739 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e37bd91-f896-41fc-ba25-4480ff576fae None admin] PATH_INFO: `/v3/users/a1613582a9844c5fa354d54653ab8e01` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:45:53.536186 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e37bd91-f896-41fc-ba25-4480ff576fae None admin] RBAC: Authorizing `identity:get_user(user_id=a1613582a9844c5fa354d54653ab8e01)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:45:53.537816 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e37bd91-f896-41fc-ba25-4480ff576fae None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:45:53.539783 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 35/67] 10.222.0.59 () {56 vars in 1177 bytes} [Tue Oct 11 11:45:53 2022] GET /identity/v3/users/a1613582a9844c5fa354d54653ab8e01 => generated 247 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:45:53.540090 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-7f7155ba-d2d0-48b2-9cd5-407dceef7063 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:45:53.541212 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e37bd91-f896-41fc-ba25-4480ff576fae None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:45:53.541335 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-7f7155ba-d2d0-48b2-9cd5-407dceef7063 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:45:53.555392 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:45:53.601605 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:45:53.624965 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 33/68] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:45:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 11:45:53.625855 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7155ba-d2d0-48b2-9cd5-407dceef7063 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:45:53.672593 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e37bd91-f896-41fc-ba25-4480ff576fae None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:45:53.718994 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb7c928-3bef-42f9-9e53-8023240d4c25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:45:53.720568 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb7c928-3bef-42f9-9e53-8023240d4c25 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:45:53.721027 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb7c928-3bef-42f9-9e53-8023240d4c25 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:45:53.721471 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb7c928-3bef-42f9-9e53-8023240d4c25 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:45:53.737699 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb7c928-3bef-42f9-9e53-8023240d4c25 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 11:45:53.747669 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application [None req-afb7c928-3bef-42f9-9e53-8023240d4c25 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 11 11:45:53.747669 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:45:53.747669 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:45:53.747669 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:45:53.747669 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:45:53.747669 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:45:53.747669 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:45:53.747669 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:45:53.747669 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:45:53.747669 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:45:53.747669 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:45:53.747669 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:45:53.747669 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 11:45:53.747669 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 11:45:53.747669 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 11:45:53.747669 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 11:45:53.748616 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 11:45:53.748616 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 11:45:53.748616 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 11:45:53.748616 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 11:45:53.748616 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:45:53.748616 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:45:53.748616 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:45:53.748616 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:45:53.748616 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:45:53.748616 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:45:53.748616 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:45:53.748616 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application with Lock( Oct 11 11:45:53.748616 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:45:53.748616 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:45:53.748616 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:45:53.748616 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:45:53.749362 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:45:53.749362 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:45:53.749362 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:45:53.749362 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:45:53.749362 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 11:45:53.749362 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 11:45:53.749362 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 11:45:53.749362 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 11:45:53.749362 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 11:45:53.749362 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 11:45:53.749362 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 11 11:45:53.749362 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:45:53.749939 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 36/69] 10.222.0.59 () {56 vars in 1114 bytes} [Tue Oct 11 11:45:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:45:53.751295 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb7c928-3bef-42f9-9e53-8023240d4c25 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:45:53.766530 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7155ba-d2d0-48b2-9cd5-407dceef7063 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:45:53.804554 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edc8b42-3555-4e46-ac8a-d8ebfacadc3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:45:53.805232 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edc8b42-3555-4e46-ac8a-d8ebfacadc3e None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:45:53.805517 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edc8b42-3555-4e46-ac8a-d8ebfacadc3e None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:45:53.805844 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edc8b42-3555-4e46-ac8a-d8ebfacadc3e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:45:53.817453 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edc8b42-3555-4e46-ac8a-d8ebfacadc3e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 11:45:53.822346 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application [None req-8edc8b42-3555-4e46-ac8a-d8ebfacadc3e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 11 11:45:53.822346 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:45:53.822346 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:45:53.822346 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:45:53.822346 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:45:53.822346 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:45:53.822346 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:45:53.822346 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:45:53.822346 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:45:53.822346 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:45:53.822346 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:45:53.822346 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:45:53.822346 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 11:45:53.822346 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 11:45:53.822346 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 11:45:53.822346 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 11:45:53.823378 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 11:45:53.823378 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 11:45:53.823378 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 11:45:53.823378 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 11:45:53.823378 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:45:53.823378 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:45:53.823378 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:45:53.823378 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:45:53.823378 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:45:53.823378 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:45:53.823378 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:45:53.823378 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application with Lock( Oct 11 11:45:53.823378 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:45:53.823378 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:45:53.823378 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:45:53.823378 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:45:53.824149 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:45:53.824149 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:45:53.824149 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:45:53.824149 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:45:53.824149 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 11:45:53.824149 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 11:45:53.824149 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 11:45:53.824149 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 11:45:53.824149 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 11:45:53.824149 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 11:45:53.824149 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 11 11:45:53.824149 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:45:53.824149 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 34/70] 10.222.0.59 () {56 vars in 1099 bytes} [Tue Oct 11 11:45:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:45:53.824780 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edc8b42-3555-4e46-ac8a-d8ebfacadc3e None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:45:53.827828 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:45:53.920993 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb7c928-3bef-42f9-9e53-8023240d4c25 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:45:53.931300 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:45:53.959180 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2003f2-26b2-49c1-b2f3-46d4ce0a62c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:45:53.959818 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2003f2-26b2-49c1-b2f3-46d4ce0a62c2 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:45:53.960103 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2003f2-26b2-49c1-b2f3-46d4ce0a62c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:45:53.960383 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2003f2-26b2-49c1-b2f3-46d4ce0a62c2 None admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:45:53.961083 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2003f2-26b2-49c1-b2f3-46d4ce0a62c2 None admin] RBAC: Adding query filter params (name=member) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:45:53.961393 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2003f2-26b2-49c1-b2f3-46d4ce0a62c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:45:53.962910 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2003f2-26b2-49c1-b2f3-46d4ce0a62c2 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:45:53.971905 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 37/71] 10.222.0.59 () {56 vars in 1101 bytes} [Tue Oct 11 11:45:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:45:53.972433 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edc8b42-3555-4e46-ac8a-d8ebfacadc3e None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:45:53.973536 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2003f2-26b2-49c1-b2f3-46d4ce0a62c2 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:45:54.016439 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db9b6c5-8b35-401d-b7e7-e49297f7cfb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:45:54.017100 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db9b6c5-8b35-401d-b7e7-e49297f7cfb5 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:45:54.017385 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db9b6c5-8b35-401d-b7e7-e49297f7cfb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:45:54.017661 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db9b6c5-8b35-401d-b7e7-e49297f7cfb5 None admin] PATH_INFO: `/v3/projects/1a4b3d97a66e4af4be0a066285f102de` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:45:54.018849 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db9b6c5-8b35-401d-b7e7-e49297f7cfb5 None admin] RBAC: Authorizing `identity:get_project(project_id=1a4b3d97a66e4af4be0a066285f102de)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:45:54.020521 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db9b6c5-8b35-401d-b7e7-e49297f7cfb5 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:45:54.022262 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 35/72] 10.222.0.59 () {56 vars in 1186 bytes} [Tue Oct 11 11:45:53 2022] GET /identity/v3/projects/1a4b3d97a66e4af4be0a066285f102de => generated 298 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:45:54.023117 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db9b6c5-8b35-401d-b7e7-e49297f7cfb5 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:45:54.092774 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2003f2-26b2-49c1-b2f3-46d4ce0a62c2 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:45:54.115414 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:45:54.144963 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd22d3d-077c-492d-b511-c0749ac7f717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:45:54.145921 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd22d3d-077c-492d-b511-c0749ac7f717 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:45:54.146415 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd22d3d-077c-492d-b511-c0749ac7f717 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:45:54.146858 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd22d3d-077c-492d-b511-c0749ac7f717 None admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:45:54.147988 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd22d3d-077c-492d-b511-c0749ac7f717 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:45:54.148513 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd22d3d-077c-492d-b511-c0749ac7f717 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:45:54.150893 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd22d3d-077c-492d-b511-c0749ac7f717 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:45:54.159929 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db9b6c5-8b35-401d-b7e7-e49297f7cfb5 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:45:54.160828 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 38/73] 10.222.0.59 () {56 vars in 1111 bytes} [Tue Oct 11 11:45:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:45:54.161686 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd22d3d-077c-492d-b511-c0749ac7f717 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:45:54.195928 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:45:54.204643 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d029b7-7939-4b6c-9c82-485d13b569b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:45:54.205600 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d029b7-7939-4b6c-9c82-485d13b569b8 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:45:54.206051 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d029b7-7939-4b6c-9c82-485d13b569b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:45:54.206580 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d029b7-7939-4b6c-9c82-485d13b569b8 None admin] PATH_INFO: `/v3/roles` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:45:54.207697 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d029b7-7939-4b6c-9c82-485d13b569b8 None admin] RBAC: Adding query filter params (name=member) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:45:54.208195 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d029b7-7939-4b6c-9c82-485d13b569b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:45:54.211100 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d029b7-7939-4b6c-9c82-485d13b569b8 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:45:54.230995 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 36/74] 10.222.0.59 () {56 vars in 1101 bytes} [Tue Oct 11 11:45:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:45:54.232308 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d029b7-7939-4b6c-9c82-485d13b569b8 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:45:54.293895 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd22d3d-077c-492d-b511-c0749ac7f717 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:45:54.331438 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4850b18-7c15-4436-810d-2c1ef9925dae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:45:54.332194 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4850b18-7c15-4436-810d-2c1ef9925dae None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:45:54.332474 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4850b18-7c15-4436-810d-2c1ef9925dae None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:45:54.332750 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4850b18-7c15-4436-810d-2c1ef9925dae None admin] PATH_INFO: `/v3/users/4965b07351a94e97bd57865e425127be` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:45:54.347955 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4850b18-7c15-4436-810d-2c1ef9925dae None admin] RBAC: Authorizing `identity:get_user(user_id=4965b07351a94e97bd57865e425127be)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:45:54.349467 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4850b18-7c15-4436-810d-2c1ef9925dae None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:45:54.351311 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 39/75] 10.222.0.59 () {56 vars in 1177 bytes} [Tue Oct 11 11:45:54 2022] GET /identity/v3/users/4965b07351a94e97bd57865e425127be => generated 275 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:45:54.352101 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4850b18-7c15-4436-810d-2c1ef9925dae None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:45:54.354651 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d029b7-7939-4b6c-9c82-485d13b569b8 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:45:54.384907 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:45:54.391171 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4341416b-ded8-43ed-822f-d25f7a8de90b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:45:54.391836 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4341416b-ded8-43ed-822f-d25f7a8de90b None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:45:54.392117 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4341416b-ded8-43ed-822f-d25f7a8de90b None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:45:54.392394 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4341416b-ded8-43ed-822f-d25f7a8de90b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:45:54.393536 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4341416b-ded8-43ed-822f-d25f7a8de90b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:45:54.395107 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4341416b-ded8-43ed-822f-d25f7a8de90b None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:45:54.396706 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 37/76] 10.222.0.59 () {56 vars in 1108 bytes} [Tue Oct 11 11:45:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:45:54.397492 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4341416b-ded8-43ed-822f-d25f7a8de90b None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:45:54.455225 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:45:54.466763 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4850b18-7c15-4436-810d-2c1ef9925dae None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:45:54.503479 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29357199-a0ee-4bfb-b9cb-d2b866e2bbef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:45:54.504155 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29357199-a0ee-4bfb-b9cb-d2b866e2bbef None admin] REQUEST_METHOD: `PUT` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:45:54.504528 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29357199-a0ee-4bfb-b9cb-d2b866e2bbef None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:45:54.504870 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29357199-a0ee-4bfb-b9cb-d2b866e2bbef None admin] PATH_INFO: `/v3/projects/1a4b3d97a66e4af4be0a066285f102de/users/a1613582a9844c5fa354d54653ab8e01/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:45:54.505265 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4341416b-ded8-43ed-822f-d25f7a8de90b None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:45:54.507092 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29357199-a0ee-4bfb-b9cb-d2b866e2bbef None admin] RBAC: Authorizing `identity:create_grant(project_id=1a4b3d97a66e4af4be0a066285f102de, user_id=a1613582a9844c5fa354d54653ab8e01, role_id=5cf1f4bd2fef46ab9c6eff56d52f9f69)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:45:54.509177 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29357199-a0ee-4bfb-b9cb-d2b866e2bbef None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:45:54.523495 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 40/77] 10.222.0.59 () {58 vars in 1439 bytes} [Tue Oct 11 11:45:54 2022] PUT /identity/v3/projects/1a4b3d97a66e4af4be0a066285f102de/users/a1613582a9844c5fa354d54653ab8e01/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 11:45:54.524356 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29357199-a0ee-4bfb-b9cb-d2b866e2bbef None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:45:54.553768 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e500e2de-a192-48c4-8e27-50fe632ffaf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:45:54.554719 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e500e2de-a192-48c4-8e27-50fe632ffaf6 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:45:54.555146 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e500e2de-a192-48c4-8e27-50fe632ffaf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:45:54.555549 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e500e2de-a192-48c4-8e27-50fe632ffaf6 None admin] PATH_INFO: `/v3/users/4965b07351a94e97bd57865e425127be` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:45:54.558006 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e500e2de-a192-48c4-8e27-50fe632ffaf6 None admin] RBAC: Authorizing `identity:get_user(user_id=4965b07351a94e97bd57865e425127be)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:45:54.560105 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e500e2de-a192-48c4-8e27-50fe632ffaf6 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:45:54.562900 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 38/78] 10.222.0.59 () {56 vars in 1177 bytes} [Tue Oct 11 11:45:54 2022] GET /identity/v3/users/4965b07351a94e97bd57865e425127be => generated 275 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:45:54.564011 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e500e2de-a192-48c4-8e27-50fe632ffaf6 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:45:54.660619 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:45:54.680246 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29357199-a0ee-4bfb-b9cb-d2b866e2bbef None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:45:54.709759 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e500e2de-a192-48c4-8e27-50fe632ffaf6 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:45:54.725586 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc089851-6bac-4802-9008-e1fc947d79bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:45:54.726595 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc089851-6bac-4802-9008-e1fc947d79bf None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:45:54.727035 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc089851-6bac-4802-9008-e1fc947d79bf None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:45:54.727464 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc089851-6bac-4802-9008-e1fc947d79bf None admin] PATH_INFO: `/v3/users/4965b07351a94e97bd57865e425127be` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:45:54.729416 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc089851-6bac-4802-9008-e1fc947d79bf None admin] RBAC: Authorizing `identity:get_user(user_id=4965b07351a94e97bd57865e425127be)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:45:54.732353 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc089851-6bac-4802-9008-e1fc947d79bf None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:45:54.735135 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 41/79] 10.222.0.59 () {56 vars in 1177 bytes} [Tue Oct 11 11:45:54 2022] GET /identity/v3/users/4965b07351a94e97bd57865e425127be => generated 275 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:45:54.736472 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc089851-6bac-4802-9008-e1fc947d79bf None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:45:54.761575 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e72cafb-4517-4f4a-a72b-bdebd279189e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:45:54.762622 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e72cafb-4517-4f4a-a72b-bdebd279189e None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:45:54.763109 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e72cafb-4517-4f4a-a72b-bdebd279189e None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:45:54.763817 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e72cafb-4517-4f4a-a72b-bdebd279189e None admin] PATH_INFO: `/v3/projects/1a4b3d97a66e4af4be0a066285f102de` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:45:54.766476 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e72cafb-4517-4f4a-a72b-bdebd279189e None admin] RBAC: Authorizing `identity:get_project(project_id=1a4b3d97a66e4af4be0a066285f102de)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:45:54.769051 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:45:54.769293 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e72cafb-4517-4f4a-a72b-bdebd279189e None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:45:54.772394 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 39/80] 10.222.0.59 () {56 vars in 1186 bytes} [Tue Oct 11 11:45:54 2022] GET /identity/v3/projects/1a4b3d97a66e4af4be0a066285f102de => generated 298 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:45:54.773825 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e72cafb-4517-4f4a-a72b-bdebd279189e None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:45:54.877758 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc089851-6bac-4802-9008-e1fc947d79bf None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:45:54.922409 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74a00e6-e7ad-4937-bb23-72eb8949735c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:45:54.923492 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74a00e6-e7ad-4937-bb23-72eb8949735c None admin] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:45:54.923994 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74a00e6-e7ad-4937-bb23-72eb8949735c None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:45:54.924471 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74a00e6-e7ad-4937-bb23-72eb8949735c None admin] PATH_INFO: `/v3/users` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:45:54.925784 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74a00e6-e7ad-4937-bb23-72eb8949735c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:45:54.928906 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74a00e6-e7ad-4937-bb23-72eb8949735c None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:45:54.929072 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e72cafb-4517-4f4a-a72b-bdebd279189e None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:45:54.960174 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 42/81] 10.222.0.59 () {60 vars in 1132 bytes} [Tue Oct 11 11:45:54 2022] POST /identity/v3/users => generated 289 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 11:45:54.961510 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74a00e6-e7ad-4937-bb23-72eb8949735c None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:45:54.996214 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc5a436-9efa-4537-adf2-ec909634b308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:45:54.996941 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc5a436-9efa-4537-adf2-ec909634b308 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:45:54.997227 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc5a436-9efa-4537-adf2-ec909634b308 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:45:54.997508 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc5a436-9efa-4537-adf2-ec909634b308 None admin] PATH_INFO: `/v3/projects/1a4b3d97a66e4af4be0a066285f102de` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:45:54.999800 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc5a436-9efa-4537-adf2-ec909634b308 None admin] RBAC: Authorizing `identity:get_project(project_id=1a4b3d97a66e4af4be0a066285f102de)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:45:55.001739 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc5a436-9efa-4537-adf2-ec909634b308 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:45:55.004619 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 40/82] 10.222.0.59 () {56 vars in 1186 bytes} [Tue Oct 11 11:45:54 2022] GET /identity/v3/projects/1a4b3d97a66e4af4be0a066285f102de => generated 298 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:45:55.005908 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc5a436-9efa-4537-adf2-ec909634b308 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:45:55.011003 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:45:55.065763 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:45:55.100252 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74a00e6-e7ad-4937-bb23-72eb8949735c None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:45:55.144629 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3117b21f-f68e-4f24-ac07-afd6246c4ee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:45:55.145681 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3117b21f-f68e-4f24-ac07-afd6246c4ee1 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:45:55.146227 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3117b21f-f68e-4f24-ac07-afd6246c4ee1 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:45:55.146744 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3117b21f-f68e-4f24-ac07-afd6246c4ee1 None admin] PATH_INFO: `/v3/projects/5d6e6621eec7434c92d9386de03b2c6e` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:45:55.148308 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3117b21f-f68e-4f24-ac07-afd6246c4ee1 None admin] RBAC: Authorizing `identity:get_project(project_id=5d6e6621eec7434c92d9386de03b2c6e)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:45:55.150120 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3117b21f-f68e-4f24-ac07-afd6246c4ee1 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:45:55.152671 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 43/83] 10.222.0.59 () {56 vars in 1186 bytes} [Tue Oct 11 11:45:54 2022] GET /identity/v3/projects/5d6e6621eec7434c92d9386de03b2c6e => generated 312 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:45:55.153885 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3117b21f-f68e-4f24-ac07-afd6246c4ee1 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:45:55.160906 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc5a436-9efa-4537-adf2-ec909634b308 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:45:55.214601 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d76d09c-413e-4bec-b5d0-ac1baa598348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:45:55.215850 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d76d09c-413e-4bec-b5d0-ac1baa598348 None admin] REQUEST_METHOD: `PUT` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:45:55.216458 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d76d09c-413e-4bec-b5d0-ac1baa598348 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:45:55.216959 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d76d09c-413e-4bec-b5d0-ac1baa598348 None admin] PATH_INFO: `/v3/projects/1a4b3d97a66e4af4be0a066285f102de/users/4965b07351a94e97bd57865e425127be/roles/de2d2bd26fff4748a79990f97a107164` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:45:55.219500 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d76d09c-413e-4bec-b5d0-ac1baa598348 None admin] RBAC: Authorizing `identity:create_grant(project_id=1a4b3d97a66e4af4be0a066285f102de, user_id=4965b07351a94e97bd57865e425127be, role_id=de2d2bd26fff4748a79990f97a107164)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:45:55.221562 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d76d09c-413e-4bec-b5d0-ac1baa598348 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:45:55.251343 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 41/84] 10.222.0.59 () {58 vars in 1439 bytes} [Tue Oct 11 11:45:55 2022] PUT /identity/v3/projects/1a4b3d97a66e4af4be0a066285f102de/users/4965b07351a94e97bd57865e425127be/roles/de2d2bd26fff4748a79990f97a107164 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 11:45:55.252569 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d76d09c-413e-4bec-b5d0-ac1baa598348 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:45:55.326864 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3117b21f-f68e-4f24-ac07-afd6246c4ee1 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:45:55.354846 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:45:55.404380 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f1603a-d116-4685-afeb-984a6d9440d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:45:55.405084 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f1603a-d116-4685-afeb-984a6d9440d9 None admin] REQUEST_METHOD: `PUT` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:45:55.405366 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f1603a-d116-4685-afeb-984a6d9440d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:45:55.408002 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f1603a-d116-4685-afeb-984a6d9440d9 None admin] PATH_INFO: `/v3/projects/1a4b3d97a66e4af4be0a066285f102de/users/4965b07351a94e97bd57865e425127be/roles/3a1a5476791547419a0b885c2022ef68` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:45:55.411142 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f1603a-d116-4685-afeb-984a6d9440d9 None admin] RBAC: Authorizing `identity:create_grant(project_id=1a4b3d97a66e4af4be0a066285f102de, user_id=4965b07351a94e97bd57865e425127be, role_id=3a1a5476791547419a0b885c2022ef68)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:45:55.412793 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f1603a-d116-4685-afeb-984a6d9440d9 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:45:55.433386 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d76d09c-413e-4bec-b5d0-ac1baa598348 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:45:55.433686 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 44/85] 10.222.0.59 () {58 vars in 1439 bytes} [Tue Oct 11 11:45:55 2022] PUT /identity/v3/projects/1a4b3d97a66e4af4be0a066285f102de/users/4965b07351a94e97bd57865e425127be/roles/3a1a5476791547419a0b885c2022ef68 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 11:45:55.513541 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf627f8-ba1f-4743-9822-819736828144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:45:55.514382 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf627f8-ba1f-4743-9822-819736828144 None admin] REQUEST_METHOD: `PUT` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:45:55.514747 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf627f8-ba1f-4743-9822-819736828144 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:45:55.514991 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf627f8-ba1f-4743-9822-819736828144 None admin] PATH_INFO: `/v3/projects/5d6e6621eec7434c92d9386de03b2c6e/users/4965b07351a94e97bd57865e425127be/roles/de2d2bd26fff4748a79990f97a107164` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:45:55.517223 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf627f8-ba1f-4743-9822-819736828144 None admin] RBAC: Authorizing `identity:create_grant(project_id=5d6e6621eec7434c92d9386de03b2c6e, user_id=4965b07351a94e97bd57865e425127be, role_id=de2d2bd26fff4748a79990f97a107164)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:45:55.518998 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf627f8-ba1f-4743-9822-819736828144 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:45:55.539853 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 42/86] 10.222.0.59 () {58 vars in 1439 bytes} [Tue Oct 11 11:45:55 2022] PUT /identity/v3/projects/5d6e6621eec7434c92d9386de03b2c6e/users/4965b07351a94e97bd57865e425127be/roles/de2d2bd26fff4748a79990f97a107164 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 11:45:57.684656 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2c79b1-8b6f-4b2c-98bb-34021daa59d7 None None] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:45:57.685534 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2c79b1-8b6f-4b2c-98bb-34021daa59d7 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:45:57.685534 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2c79b1-8b6f-4b2c-98bb-34021daa59d7 None None] PATH_INFO: `/` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:45:57.687266 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 45/87] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:45:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 11 11:45:57.704786 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b6b98d-ded2-45bd-bc53-2a6857d05410 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:45:57.705532 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b6b98d-ded2-45bd-bc53-2a6857d05410 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:45:57.705532 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b6b98d-ded2-45bd-bc53-2a6857d05410 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:45:57.729928 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-85b6b98d-ded2-45bd-bc53-2a6857d05410 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:45:57.731271 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-85b6b98d-ded2-45bd-bc53-2a6857d05410 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:45:57.825269 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 43/88] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:45:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 11 11:45:58.371992 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2c79b1-8b6f-4b2c-98bb-34021daa59d7 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:45:58.375364 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:45:58.554476 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2c79b1-8b6f-4b2c-98bb-34021daa59d7 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:45:58.640385 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d4174f-3e20-45da-bc91-6e20ac4a2433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:45:58.641878 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d4174f-3e20-45da-bc91-6e20ac4a2433 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:45:58.642647 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d4174f-3e20-45da-bc91-6e20ac4a2433 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:45:58.643417 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d4174f-3e20-45da-bc91-6e20ac4a2433 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:45:58.673701 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d4174f-3e20-45da-bc91-6e20ac4a2433 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 11:45:58.684038 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application [None req-a9d4174f-3e20-45da-bc91-6e20ac4a2433 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 11 11:45:58.684038 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:45:58.684038 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:45:58.684038 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:45:58.684038 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:45:58.684038 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:45:58.684038 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:45:58.684038 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:45:58.684038 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:45:58.684038 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:45:58.684038 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:45:58.684038 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:45:58.684038 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 11:45:58.684038 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 11:45:58.684038 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 11:45:58.684038 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 11:45:58.686445 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 11:45:58.686445 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 11:45:58.686445 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 11:45:58.686445 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 11:45:58.686445 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:45:58.686445 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:45:58.686445 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:45:58.686445 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:45:58.686445 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:45:58.686445 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:45:58.686445 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:45:58.686445 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application with Lock( Oct 11 11:45:58.686445 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:45:58.686445 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:45:58.686445 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:45:58.686445 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:45:58.687909 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:45:58.687909 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:45:58.687909 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:45:58.687909 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:45:58.687909 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 11:45:58.687909 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 11:45:58.687909 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 11:45:58.687909 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 11:45:58.687909 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 11:45:58.687909 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 11:45:58.687909 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 11 11:45:58.687909 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:45:58.696090 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 46/89] 10.222.0.59 () {56 vars in 1096 bytes} [Tue Oct 11 11:45:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 11 11:45:58.703784 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b6b98d-ded2-45bd-bc53-2a6857d05410 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:45:58.789824 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7ac95e-339a-4195-9c66-c74473fe7068 None None] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:45:58.790789 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7ac95e-339a-4195-9c66-c74473fe7068 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:45:58.791032 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:45:58.792403 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7ac95e-339a-4195-9c66-c74473fe7068 None None] PATH_INFO: `/` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:45:58.796131 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 47/90] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:45:58 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:45:58.805692 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4835dc6e-6752-49d2-be37-88f42faa70a0 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:45:58.806593 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4835dc6e-6752-49d2-be37-88f42faa70a0 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:45:58.807311 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4835dc6e-6752-49d2-be37-88f42faa70a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:45:58.844738 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-4835dc6e-6752-49d2-be37-88f42faa70a0 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:45:58.846408 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-4835dc6e-6752-49d2-be37-88f42faa70a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:45:58.928765 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b6b98d-ded2-45bd-bc53-2a6857d05410 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:45:58.980388 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:45:58.996326 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 48/91] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:45:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 11 11:45:58.998678 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ed8e4d-1eb5-43db-8f93-69f26a5a8a33 None None] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:45:58.999183 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ed8e4d-1eb5-43db-8f93-69f26a5a8a33 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:45:58.999679 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ed8e4d-1eb5-43db-8f93-69f26a5a8a33 None None] PATH_INFO: `/` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:45:59.002439 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 49/92] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:45:58 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:45:59.008886 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937359e9-3732-4449-a645-3daf68851386 None None] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:45:59.009387 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937359e9-3732-4449-a645-3daf68851386 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:45:59.009888 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937359e9-3732-4449-a645-3daf68851386 None None] PATH_INFO: `/` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:45:59.014920 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 50/93] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:45:59 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:45:59.017613 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150d5050-b0d8-4d38-9178-5589f03cdace None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:45:59.018970 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150d5050-b0d8-4d38-9178-5589f03cdace None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:45:59.019536 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150d5050-b0d8-4d38-9178-5589f03cdace None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:45:59.029146 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cce4c1-fba1-4f22-aea1-14e722076a0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:45:59.030319 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cce4c1-fba1-4f22-aea1-14e722076a0e None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:45:59.030850 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cce4c1-fba1-4f22-aea1-14e722076a0e None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:45:59.031326 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cce4c1-fba1-4f22-aea1-14e722076a0e None admin] PATH_INFO: `/v3/roles` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:45:59.032651 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cce4c1-fba1-4f22-aea1-14e722076a0e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:45:59.033174 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cce4c1-fba1-4f22-aea1-14e722076a0e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:45:59.036057 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cce4c1-fba1-4f22-aea1-14e722076a0e None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:45:59.052815 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-150d5050-b0d8-4d38-9178-5589f03cdace None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:45:59.055550 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-150d5050-b0d8-4d38-9178-5589f03cdace None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:45:59.064355 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 44/94] 10.222.0.59 () {56 vars in 1099 bytes} [Tue Oct 11 11:45:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 361 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:45:59.066540 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7ce9d6-7e55-4cec-b503-cf03650ac88e None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:45:59.067058 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7ce9d6-7e55-4cec-b503-cf03650ac88e None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:45:59.067539 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7ce9d6-7e55-4cec-b503-cf03650ac88e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:45:59.113621 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-ff7ce9d6-7e55-4cec-b503-cf03650ac88e None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:45:59.115212 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-ff7ce9d6-7e55-4cec-b503-cf03650ac88e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:45:59.201047 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 51/95] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:45:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 11 11:45:59.203715 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150d5050-b0d8-4d38-9178-5589f03cdace None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:45:59.244772 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 45/96] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:45:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 11:45:59.356515 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150d5050-b0d8-4d38-9178-5589f03cdace None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:45:59.389828 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:45:59.408065 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e1fed8-44f6-4370-8d01-9845bf984c77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:45:59.409119 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e1fed8-44f6-4370-8d01-9845bf984c77 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:45:59.409955 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e1fed8-44f6-4370-8d01-9845bf984c77 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:45:59.410427 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e1fed8-44f6-4370-8d01-9845bf984c77 None admin] PATH_INFO: `/v3/users/a1613582a9844c5fa354d54653ab8e01` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:45:59.411834 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e1fed8-44f6-4370-8d01-9845bf984c77 None admin] RBAC: Authorizing `identity:get_user(user_id=a1613582a9844c5fa354d54653ab8e01)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:45:59.413553 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e1fed8-44f6-4370-8d01-9845bf984c77 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:45:59.415831 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 52/97] 10.222.0.59 () {56 vars in 1177 bytes} [Tue Oct 11 11:45:59 2022] GET /identity/v3/users/a1613582a9844c5fa354d54653ab8e01 => generated 247 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:45:59.421160 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7ce9d6-7e55-4cec-b503-cf03650ac88e None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:45:59.480395 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:45:59.602089 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7ce9d6-7e55-4cec-b503-cf03650ac88e None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:45:59.677921 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0208d5e9-9cc9-4032-8217-123a134eecd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:45:59.678997 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0208d5e9-9cc9-4032-8217-123a134eecd4 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:45:59.679482 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0208d5e9-9cc9-4032-8217-123a134eecd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:45:59.679976 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0208d5e9-9cc9-4032-8217-123a134eecd4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:45:59.729293 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e1fed8-44f6-4370-8d01-9845bf984c77 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:45:59.838400 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0208d5e9-9cc9-4032-8217-123a134eecd4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:45:59.840700 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0208d5e9-9cc9-4032-8217-123a134eecd4 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:45:59.897457 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e1fed8-44f6-4370-8d01-9845bf984c77 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:45:59.900025 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:45:59.946017 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ace7c7c-db8c-42ad-94ef-d7bc7d03ff4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:45:59.946912 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ace7c7c-db8c-42ad-94ef-d7bc7d03ff4f None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:45:59.947391 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ace7c7c-db8c-42ad-94ef-d7bc7d03ff4f None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:45:59.947875 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ace7c7c-db8c-42ad-94ef-d7bc7d03ff4f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:45:59.961270 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ace7c7c-db8c-42ad-94ef-d7bc7d03ff4f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 11:45:59.970661 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application [None req-1ace7c7c-db8c-42ad-94ef-d7bc7d03ff4f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 11 11:45:59.970661 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:45:59.970661 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:45:59.970661 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:45:59.970661 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:45:59.970661 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:45:59.970661 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:45:59.970661 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:45:59.970661 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:45:59.970661 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:45:59.970661 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:45:59.970661 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:45:59.970661 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 11:45:59.970661 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 11:45:59.970661 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 11:45:59.970661 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 11:45:59.972410 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 11:45:59.972410 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 11:45:59.972410 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 11:45:59.972410 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 11:45:59.972410 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:45:59.972410 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:45:59.972410 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:45:59.972410 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:45:59.972410 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:45:59.972410 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:45:59.972410 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:45:59.972410 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application with Lock( Oct 11 11:45:59.972410 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:45:59.972410 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:45:59.972410 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:45:59.972410 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:45:59.973888 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:45:59.973888 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:45:59.973888 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:45:59.973888 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:45:59.973888 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 11:45:59.973888 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 11:45:59.973888 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 11:45:59.973888 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 11:45:59.973888 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 11:45:59.973888 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 11:45:59.973888 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 11 11:45:59.973888 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:45:59.973888 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 53/98] 10.222.0.59 () {56 vars in 1099 bytes} [Tue Oct 11 11:45:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:45:59.977149 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dbfba4-5c66-4d1b-af63-9f4329dc6c48 None None] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:45:59.977149 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dbfba4-5c66-4d1b-af63-9f4329dc6c48 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:45:59.977149 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dbfba4-5c66-4d1b-af63-9f4329dc6c48 None None] PATH_INFO: `/` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:45:59.982607 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 54/99] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:45:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:45:59.982607 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dbfba4-5c66-4d1b-af63-9f4329dc6c48 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:00.016392 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 46/100] 10.222.0.59 () {58 vars in 1282 bytes} [Tue Oct 11 11:45:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 594 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 11 11:46:00.017285 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0208d5e9-9cc9-4032-8217-123a134eecd4 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:00.058482 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:00.145637 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dbfba4-5c66-4d1b-af63-9f4329dc6c48 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:00.164894 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0208d5e9-9cc9-4032-8217-123a134eecd4 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:00.190383 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901b11e8-7c34-4410-bdb9-97dad1292c1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:00.191341 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901b11e8-7c34-4410-bdb9-97dad1292c1d None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:00.191816 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901b11e8-7c34-4410-bdb9-97dad1292c1d None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:00.192221 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901b11e8-7c34-4410-bdb9-97dad1292c1d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:00.211188 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901b11e8-7c34-4410-bdb9-97dad1292c1d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 11:46:00.221803 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application [None req-901b11e8-7c34-4410-bdb9-97dad1292c1d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 11 11:46:00.221803 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:46:00.221803 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:46:00.221803 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:46:00.221803 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:46:00.221803 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:46:00.221803 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:46:00.221803 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:46:00.221803 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:46:00.221803 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:46:00.221803 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:46:00.221803 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:46:00.221803 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 11:46:00.221803 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 11:46:00.221803 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 11:46:00.221803 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 11:46:00.223912 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 11:46:00.223912 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 11:46:00.223912 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 11:46:00.223912 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 11:46:00.223912 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:46:00.223912 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:46:00.223912 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:46:00.223912 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:46:00.223912 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:46:00.223912 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:46:00.223912 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:46:00.223912 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application with Lock( Oct 11 11:46:00.223912 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:46:00.223912 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:46:00.223912 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:46:00.223912 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:46:00.225301 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:46:00.225301 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:46:00.225301 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:46:00.225301 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:46:00.225301 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 11:46:00.225301 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 11:46:00.225301 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 11:46:00.225301 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 11:46:00.225301 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 11:46:00.225301 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 11:46:00.225301 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 11 11:46:00.225301 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:46:00.225881 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d28dcf3-a771-4a61-9d55-b86da6f29ae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:00.225881 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d28dcf3-a771-4a61-9d55-b86da6f29ae9 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:00.225881 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d28dcf3-a771-4a61-9d55-b86da6f29ae9 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:00.226076 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 55/101] 10.222.0.59 () {56 vars in 1099 bytes} [Tue Oct 11 11:45:59 2022] GET /identity/v3/roles/reader => generated 84 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:46:00.226139 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d28dcf3-a771-4a61-9d55-b86da6f29ae9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:00.226260 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901b11e8-7c34-4410-bdb9-97dad1292c1d None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:00.241056 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d28dcf3-a771-4a61-9d55-b86da6f29ae9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 11:46:00.246110 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application [None req-3d28dcf3-a771-4a61-9d55-b86da6f29ae9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 11 11:46:00.246110 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:46:00.246110 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:46:00.246110 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:46:00.246110 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:46:00.246110 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:46:00.246110 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:46:00.246110 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:46:00.246110 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:46:00.246110 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:46:00.246110 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:46:00.246110 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:46:00.246110 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 11:46:00.246110 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 11:46:00.246110 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 11:46:00.246110 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 11:46:00.247646 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 11:46:00.247646 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 11:46:00.247646 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 11:46:00.247646 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 11:46:00.247646 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:46:00.247646 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:46:00.247646 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:46:00.247646 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:46:00.247646 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:46:00.247646 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:46:00.247646 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:46:00.247646 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application with Lock( Oct 11 11:46:00.247646 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:46:00.247646 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:46:00.247646 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:46:00.247646 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:46:00.249365 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:46:00.249365 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:46:00.249365 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:46:00.249365 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:46:00.249365 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 11:46:00.249365 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 11:46:00.249365 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 11:46:00.249365 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 11:46:00.249365 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 11:46:00.249365 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 11:46:00.249365 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 11 11:46:00.249365 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:46:00.249365 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 47/102] 10.222.0.59 () {56 vars in 1114 bytes} [Tue Oct 11 11:46:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:46:00.249365 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa515dd-86e9-40aa-ac2d-43bc2ae2cadb None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:00.250860 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa515dd-86e9-40aa-ac2d-43bc2ae2cadb None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:00.250860 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa515dd-86e9-40aa-ac2d-43bc2ae2cadb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:00.263743 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:00.274887 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-7fa515dd-86e9-40aa-ac2d-43bc2ae2cadb None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:46:00.277074 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-7fa515dd-86e9-40aa-ac2d-43bc2ae2cadb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:46:00.365278 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901b11e8-7c34-4410-bdb9-97dad1292c1d None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:00.387862 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 48/103] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:46:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 11:46:00.388935 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa515dd-86e9-40aa-ac2d-43bc2ae2cadb None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:00.389493 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:00.438627 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67afd8b-ba1d-4b7f-a75a-80f79efc3625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:00.439240 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67afd8b-ba1d-4b7f-a75a-80f79efc3625 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:00.439583 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67afd8b-ba1d-4b7f-a75a-80f79efc3625 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:00.439927 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67afd8b-ba1d-4b7f-a75a-80f79efc3625 None admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:00.441518 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67afd8b-ba1d-4b7f-a75a-80f79efc3625 None admin] RBAC: Adding query filter params (name=member) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:46:00.441518 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67afd8b-ba1d-4b7f-a75a-80f79efc3625 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:00.442669 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67afd8b-ba1d-4b7f-a75a-80f79efc3625 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:00.458594 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 56/104] 10.222.0.59 () {56 vars in 1101 bytes} [Tue Oct 11 11:46:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:46:00.458594 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67afd8b-ba1d-4b7f-a75a-80f79efc3625 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:00.558065 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa515dd-86e9-40aa-ac2d-43bc2ae2cadb None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:00.613030 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67afd8b-ba1d-4b7f-a75a-80f79efc3625 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:00.622423 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bd178f-b124-4137-8338-bfb39fd264c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:00.623551 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bd178f-b124-4137-8338-bfb39fd264c3 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:00.624055 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bd178f-b124-4137-8338-bfb39fd264c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:00.624535 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bd178f-b124-4137-8338-bfb39fd264c3 None admin] PATH_INFO: `/v3/projects/1a4b3d97a66e4af4be0a066285f102de` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:00.626520 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bd178f-b124-4137-8338-bfb39fd264c3 None admin] RBAC: Authorizing `identity:get_project(project_id=1a4b3d97a66e4af4be0a066285f102de)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:00.629304 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bd178f-b124-4137-8338-bfb39fd264c3 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:00.632372 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 49/105] 10.222.0.59 () {56 vars in 1186 bytes} [Tue Oct 11 11:46:00 2022] GET /identity/v3/projects/1a4b3d97a66e4af4be0a066285f102de => generated 298 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:46:00.633765 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bd178f-b124-4137-8338-bfb39fd264c3 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:00.641484 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:00.665559 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cbe769-5946-4eab-9336-b376dfdfef10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:00.666402 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cbe769-5946-4eab-9336-b376dfdfef10 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:00.666883 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cbe769-5946-4eab-9336-b376dfdfef10 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:00.667305 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cbe769-5946-4eab-9336-b376dfdfef10 None admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:00.668174 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cbe769-5946-4eab-9336-b376dfdfef10 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:46:00.668649 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cbe769-5946-4eab-9336-b376dfdfef10 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:00.670808 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cbe769-5946-4eab-9336-b376dfdfef10 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:00.680937 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 57/106] 10.222.0.59 () {56 vars in 1101 bytes} [Tue Oct 11 11:46:00 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:46:00.686469 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cbe769-5946-4eab-9336-b376dfdfef10 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:00.728116 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:00.816839 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cbe769-5946-4eab-9336-b376dfdfef10 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:00.822070 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bd178f-b124-4137-8338-bfb39fd264c3 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:00.857693 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731b4bbc-fd0e-408f-9709-e4a92cfbf363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:00.858517 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731b4bbc-fd0e-408f-9709-e4a92cfbf363 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:00.858960 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731b4bbc-fd0e-408f-9709-e4a92cfbf363 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:00.859354 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731b4bbc-fd0e-408f-9709-e4a92cfbf363 None admin] PATH_INFO: `/v3/users/4965b07351a94e97bd57865e425127be` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:00.860708 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731b4bbc-fd0e-408f-9709-e4a92cfbf363 None admin] RBAC: Authorizing `identity:get_user(user_id=4965b07351a94e97bd57865e425127be)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:00.862445 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731b4bbc-fd0e-408f-9709-e4a92cfbf363 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:00.863216 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085c7eb3-1f34-45c7-81e4-c1a680eb619b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:00.863932 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085c7eb3-1f34-45c7-81e4-c1a680eb619b None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:00.864251 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085c7eb3-1f34-45c7-81e4-c1a680eb619b None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:00.864596 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085c7eb3-1f34-45c7-81e4-c1a680eb619b None admin] PATH_INFO: `/v3/roles` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:00.872672 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 58/107] 10.222.0.59 () {56 vars in 1177 bytes} [Tue Oct 11 11:46:00 2022] GET /identity/v3/users/4965b07351a94e97bd57865e425127be => generated 275 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:46:00.872869 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085c7eb3-1f34-45c7-81e4-c1a680eb619b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:46:00.872869 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085c7eb3-1f34-45c7-81e4-c1a680eb619b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:00.872999 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db33ff5b-8a8c-449b-a716-f616f57dc290 None None] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:00.873063 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085c7eb3-1f34-45c7-81e4-c1a680eb619b None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:00.873126 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db33ff5b-8a8c-449b-a716-f616f57dc290 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:00.873126 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db33ff5b-8a8c-449b-a716-f616f57dc290 None None] PATH_INFO: `/` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:00.873126 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 59/108] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:46:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:46:00.873126 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db33ff5b-8a8c-449b-a716-f616f57dc290 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:00.880557 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 50/109] 10.222.0.59 () {56 vars in 1111 bytes} [Tue Oct 11 11:46:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:46:00.881779 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085c7eb3-1f34-45c7-81e4-c1a680eb619b None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:00.972687 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:01.003441 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db33ff5b-8a8c-449b-a716-f616f57dc290 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:01.005220 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:01.013492 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085c7eb3-1f34-45c7-81e4-c1a680eb619b None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:01.051460 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b451b94-1509-4367-93aa-afea4ef8ca56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:01.052241 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b451b94-1509-4367-93aa-afea4ef8ca56 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:01.052643 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b451b94-1509-4367-93aa-afea4ef8ca56 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:01.053031 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b451b94-1509-4367-93aa-afea4ef8ca56 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:01.053951 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b451b94-1509-4367-93aa-afea4ef8ca56 None admin] RBAC: Adding query filter params (role.id=5cf1f4bd2fef46ab9c6eff56d52f9f69, scope.project.id=1a4b3d97a66e4af4be0a066285f102de, user.id=a1613582a9844c5fa354d54653ab8e01) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:46:01.054367 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b451b94-1509-4367-93aa-afea4ef8ca56 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:01.056145 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b451b94-1509-4367-93aa-afea4ef8ca56 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:01.059822 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddbe786-8fec-435a-842e-5e2869baa557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:01.060602 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddbe786-8fec-435a-842e-5e2869baa557 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:01.060974 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddbe786-8fec-435a-842e-5e2869baa557 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:01.061365 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddbe786-8fec-435a-842e-5e2869baa557 None admin] PATH_INFO: `/v3/users/dd79d21ab0b94e9f9d5b6baae3791a5e` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:01.064739 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 60/110] 10.222.0.59 () {56 vars in 1374 bytes} [Tue Oct 11 11:46:00 2022] GET /identity/v3/role_assignments?user.id=a1613582a9844c5fa354d54653ab8e01&scope.project.id=1a4b3d97a66e4af4be0a066285f102de&role.id=5cf1f4bd2fef46ab9c6eff56d52f9f69 => generated 608 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:46:01.066003 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b451b94-1509-4367-93aa-afea4ef8ca56 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:01.088168 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddbe786-8fec-435a-842e-5e2869baa557 None admin] RBAC: Authorizing `identity:get_user(user_id=dd79d21ab0b94e9f9d5b6baae3791a5e)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:01.090671 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddbe786-8fec-435a-842e-5e2869baa557 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:01.093729 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 51/111] 10.222.0.59 () {56 vars in 1177 bytes} [Tue Oct 11 11:46:00 2022] GET /identity/v3/users/dd79d21ab0b94e9f9d5b6baae3791a5e => generated 289 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:46:01.094938 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddbe786-8fec-435a-842e-5e2869baa557 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:01.207065 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b451b94-1509-4367-93aa-afea4ef8ca56 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:01.258332 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d713af71-bd08-405d-b78d-96713b6a829f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:01.259392 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d713af71-bd08-405d-b78d-96713b6a829f None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:01.259818 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d713af71-bd08-405d-b78d-96713b6a829f None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:01.260346 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d713af71-bd08-405d-b78d-96713b6a829f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:01.275157 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddbe786-8fec-435a-842e-5e2869baa557 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:01.280091 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d713af71-bd08-405d-b78d-96713b6a829f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 11:46:01.289784 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application [None req-d713af71-bd08-405d-b78d-96713b6a829f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 11 11:46:01.289784 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:46:01.289784 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:46:01.289784 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:46:01.289784 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:46:01.289784 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:46:01.289784 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:46:01.289784 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:46:01.289784 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:46:01.289784 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:46:01.289784 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:46:01.289784 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:46:01.289784 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 11:46:01.289784 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 11:46:01.289784 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 11:46:01.289784 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 11:46:01.290807 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 11:46:01.290807 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 11:46:01.290807 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 11:46:01.290807 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 11:46:01.290807 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:46:01.290807 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:46:01.290807 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:46:01.290807 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:46:01.290807 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:46:01.290807 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:46:01.290807 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:46:01.290807 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application with Lock( Oct 11 11:46:01.290807 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:46:01.290807 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:46:01.290807 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:46:01.290807 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:46:01.291572 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:46:01.291572 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:46:01.291572 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:46:01.291572 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:46:01.291572 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 11:46:01.291572 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 11:46:01.291572 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 11:46:01.291572 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 11:46:01.291572 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 11:46:01.291572 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 11:46:01.291572 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 11 11:46:01.291572 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:46:01.292162 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 61/112] 10.222.0.59 () {56 vars in 1099 bytes} [Tue Oct 11 11:46:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:46:01.293873 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b72df39-aeec-41fa-8e95-9024ae5ef0b6 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:01.294300 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b72df39-aeec-41fa-8e95-9024ae5ef0b6 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:01.294760 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b72df39-aeec-41fa-8e95-9024ae5ef0b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:01.317621 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:01.325610 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-7b72df39-aeec-41fa-8e95-9024ae5ef0b6 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:46:01.327412 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-7b72df39-aeec-41fa-8e95-9024ae5ef0b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:46:01.333894 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e500cd-b556-4539-8d41-181543161aec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:01.334536 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e500cd-b556-4539-8d41-181543161aec None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:01.334853 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e500cd-b556-4539-8d41-181543161aec None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:01.335137 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e500cd-b556-4539-8d41-181543161aec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:01.366357 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:01.444669 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e500cd-b556-4539-8d41-181543161aec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:01.446272 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e500cd-b556-4539-8d41-181543161aec None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:01.446979 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 62/113] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:46:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 11:46:01.449297 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b72df39-aeec-41fa-8e95-9024ae5ef0b6 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:01.597591 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b72df39-aeec-41fa-8e95-9024ae5ef0b6 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:01.609505 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 52/114] 10.222.0.59 () {58 vars in 1282 bytes} [Tue Oct 11 11:46:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 509 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 11 11:46:01.609505 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e500cd-b556-4539-8d41-181543161aec None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:01.647980 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:01.661859 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980e1ad1-c5fa-4439-9162-e3c851c4e939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:01.662922 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980e1ad1-c5fa-4439-9162-e3c851c4e939 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:01.663371 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980e1ad1-c5fa-4439-9162-e3c851c4e939 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:01.663817 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980e1ad1-c5fa-4439-9162-e3c851c4e939 None admin] PATH_INFO: `/v3/users/4965b07351a94e97bd57865e425127be` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:01.666704 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980e1ad1-c5fa-4439-9162-e3c851c4e939 None admin] RBAC: Authorizing `identity:get_user(user_id=4965b07351a94e97bd57865e425127be)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:01.667193 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980e1ad1-c5fa-4439-9162-e3c851c4e939 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:01.677095 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 63/115] 10.222.0.59 () {56 vars in 1177 bytes} [Tue Oct 11 11:46:01 2022] GET /identity/v3/users/4965b07351a94e97bd57865e425127be => generated 275 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:46:01.677095 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980e1ad1-c5fa-4439-9162-e3c851c4e939 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:01.714804 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:01.776864 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e500cd-b556-4539-8d41-181543161aec None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:01.826549 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4785bb34-3abe-4b1b-8369-5fcf10852e45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:01.827237 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4785bb34-3abe-4b1b-8369-5fcf10852e45 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:01.827521 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4785bb34-3abe-4b1b-8369-5fcf10852e45 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:01.827797 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4785bb34-3abe-4b1b-8369-5fcf10852e45 None admin] PATH_INFO: `/v3/projects/1a4b3d97a66e4af4be0a066285f102de` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:01.829009 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4785bb34-3abe-4b1b-8369-5fcf10852e45 None admin] RBAC: Authorizing `identity:get_project(project_id=1a4b3d97a66e4af4be0a066285f102de)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:01.830513 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4785bb34-3abe-4b1b-8369-5fcf10852e45 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:01.832234 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980e1ad1-c5fa-4439-9162-e3c851c4e939 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:01.832950 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 53/116] 10.222.0.59 () {56 vars in 1186 bytes} [Tue Oct 11 11:46:01 2022] GET /identity/v3/projects/1a4b3d97a66e4af4be0a066285f102de => generated 298 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:46:01.836138 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4785bb34-3abe-4b1b-8369-5fcf10852e45 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:01.871898 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd41838-745c-4575-8d15-fb3477d682c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:01.873228 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd41838-745c-4575-8d15-fb3477d682c0 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:01.873645 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd41838-745c-4575-8d15-fb3477d682c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:01.874144 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd41838-745c-4575-8d15-fb3477d682c0 None admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:01.875379 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd41838-745c-4575-8d15-fb3477d682c0 None admin] RBAC: Adding query filter params (name=member) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:46:01.875819 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd41838-745c-4575-8d15-fb3477d682c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:01.878528 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd41838-745c-4575-8d15-fb3477d682c0 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:01.893492 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 64/117] 10.222.0.59 () {56 vars in 1101 bytes} [Tue Oct 11 11:46:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:46:01.895007 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd41838-745c-4575-8d15-fb3477d682c0 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:01.963322 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:01.975655 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4785bb34-3abe-4b1b-8369-5fcf10852e45 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:02.021502 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca02808-45e9-4eee-b67f-b6d47f5030ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:02.022673 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca02808-45e9-4eee-b67f-b6d47f5030ae None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:02.023172 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca02808-45e9-4eee-b67f-b6d47f5030ae None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:02.023680 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca02808-45e9-4eee-b67f-b6d47f5030ae None admin] PATH_INFO: `/v3/projects/5d6e6621eec7434c92d9386de03b2c6e` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:02.025862 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca02808-45e9-4eee-b67f-b6d47f5030ae None admin] RBAC: Authorizing `identity:get_project(project_id=5d6e6621eec7434c92d9386de03b2c6e)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:02.029301 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:02.029574 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca02808-45e9-4eee-b67f-b6d47f5030ae None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:02.033380 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 54/118] 10.222.0.59 () {56 vars in 1186 bytes} [Tue Oct 11 11:46:01 2022] GET /identity/v3/projects/5d6e6621eec7434c92d9386de03b2c6e => generated 312 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:46:02.035519 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca02808-45e9-4eee-b67f-b6d47f5030ae None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:02.052253 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd41838-745c-4575-8d15-fb3477d682c0 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:02.110719 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940386cf-bec3-4a16-bcc0-b94972e41828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:02.111436 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940386cf-bec3-4a16-bcc0-b94972e41828 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:02.111717 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940386cf-bec3-4a16-bcc0-b94972e41828 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:02.112117 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940386cf-bec3-4a16-bcc0-b94972e41828 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:02.167319 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca02808-45e9-4eee-b67f-b6d47f5030ae None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:02.201618 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940386cf-bec3-4a16-bcc0-b94972e41828 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:02.203182 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940386cf-bec3-4a16-bcc0-b94972e41828 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:02.207105 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12c2f10-ce71-4859-9f7f-4b23a642e38a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:02.207746 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12c2f10-ce71-4859-9f7f-4b23a642e38a None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:02.208026 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12c2f10-ce71-4859-9f7f-4b23a642e38a None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:02.208301 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12c2f10-ce71-4859-9f7f-4b23a642e38a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:02.209454 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12c2f10-ce71-4859-9f7f-4b23a642e38a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:02.210971 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12c2f10-ce71-4859-9f7f-4b23a642e38a None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:02.212622 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 55/119] 10.222.0.59 () {56 vars in 1108 bytes} [Tue Oct 11 11:46:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:46:02.213412 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12c2f10-ce71-4859-9f7f-4b23a642e38a None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:02.255751 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:02.299002 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:02.318605 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 65/120] 10.222.0.59 () {58 vars in 1282 bytes} [Tue Oct 11 11:46:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 11 11:46:02.319485 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940386cf-bec3-4a16-bcc0-b94972e41828 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:02.355119 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12c2f10-ce71-4859-9f7f-4b23a642e38a None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:02.401422 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d85c21-e5f5-45ab-a367-a244c5f680da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:02.402524 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d85c21-e5f5-45ab-a367-a244c5f680da None admin] REQUEST_METHOD: `PUT` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:02.402819 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d85c21-e5f5-45ab-a367-a244c5f680da None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:02.403094 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d85c21-e5f5-45ab-a367-a244c5f680da None admin] PATH_INFO: `/v3/projects/1a4b3d97a66e4af4be0a066285f102de/users/dd79d21ab0b94e9f9d5b6baae3791a5e/roles/acda9dcd3a754265af99c2abc579d2da` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:02.405441 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d85c21-e5f5-45ab-a367-a244c5f680da None admin] RBAC: Authorizing `identity:create_grant(project_id=1a4b3d97a66e4af4be0a066285f102de, user_id=dd79d21ab0b94e9f9d5b6baae3791a5e, role_id=acda9dcd3a754265af99c2abc579d2da)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:02.407109 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d85c21-e5f5-45ab-a367-a244c5f680da None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:02.425808 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 56/121] 10.222.0.59 () {58 vars in 1439 bytes} [Tue Oct 11 11:46:02 2022] PUT /identity/v3/projects/1a4b3d97a66e4af4be0a066285f102de/users/dd79d21ab0b94e9f9d5b6baae3791a5e/roles/acda9dcd3a754265af99c2abc579d2da => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 11:46:02.428883 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d85c21-e5f5-45ab-a367-a244c5f680da None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:02.482712 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940386cf-bec3-4a16-bcc0-b94972e41828 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:02.532586 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc6db5a-7703-418e-a66e-f881696ad70e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:02.533678 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc6db5a-7703-418e-a66e-f881696ad70e None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:02.533966 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc6db5a-7703-418e-a66e-f881696ad70e None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:02.534242 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc6db5a-7703-418e-a66e-f881696ad70e None admin] PATH_INFO: `/v3/users/4965b07351a94e97bd57865e425127be` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:02.535956 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc6db5a-7703-418e-a66e-f881696ad70e None admin] RBAC: Authorizing `identity:get_user(user_id=4965b07351a94e97bd57865e425127be)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:02.537607 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc6db5a-7703-418e-a66e-f881696ad70e None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:02.547029 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 66/122] 10.222.0.59 () {56 vars in 1177 bytes} [Tue Oct 11 11:46:02 2022] GET /identity/v3/users/4965b07351a94e97bd57865e425127be => generated 275 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:46:02.547029 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc6db5a-7703-418e-a66e-f881696ad70e None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:02.627052 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:02.645532 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:02.649925 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d85c21-e5f5-45ab-a367-a244c5f680da None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:02.710655 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294e1f16-933a-4d9c-b5fd-e9b02adba57d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:02.711745 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294e1f16-933a-4d9c-b5fd-e9b02adba57d None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:02.712263 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294e1f16-933a-4d9c-b5fd-e9b02adba57d None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:02.712762 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294e1f16-933a-4d9c-b5fd-e9b02adba57d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:02.714249 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294e1f16-933a-4d9c-b5fd-e9b02adba57d None admin] RBAC: Adding query filter params (role.id=de2d2bd26fff4748a79990f97a107164, scope.project.id=5d6e6621eec7434c92d9386de03b2c6e, user.id=4965b07351a94e97bd57865e425127be) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:46:02.714874 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294e1f16-933a-4d9c-b5fd-e9b02adba57d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:02.717908 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294e1f16-933a-4d9c-b5fd-e9b02adba57d None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:02.727993 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 57/123] 10.222.0.59 () {56 vars in 1374 bytes} [Tue Oct 11 11:46:02 2022] GET /identity/v3/role_assignments?user.id=4965b07351a94e97bd57865e425127be&scope.project.id=5d6e6621eec7434c92d9386de03b2c6e&role.id=de2d2bd26fff4748a79990f97a107164 => generated 608 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:46:02.729454 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294e1f16-933a-4d9c-b5fd-e9b02adba57d None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:02.732894 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc6db5a-7703-418e-a66e-f881696ad70e None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:02.807815 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fd7f3a-acbe-40d7-bddc-6ee9e37d6984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:02.809052 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fd7f3a-acbe-40d7-bddc-6ee9e37d6984 None admin] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:02.809646 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fd7f3a-acbe-40d7-bddc-6ee9e37d6984 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:02.810219 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fd7f3a-acbe-40d7-bddc-6ee9e37d6984 None admin] PATH_INFO: `/v3/projects` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:02.811804 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fd7f3a-acbe-40d7-bddc-6ee9e37d6984 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:02.814806 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fd7f3a-acbe-40d7-bddc-6ee9e37d6984 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:02.870069 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 67/124] 10.222.0.59 () {60 vars in 1141 bytes} [Tue Oct 11 11:46:02 2022] POST /identity/v3/projects => generated 302 bytes in 330 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 11:46:02.876590 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fd7f3a-acbe-40d7-bddc-6ee9e37d6984 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:02.938092 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294e1f16-933a-4d9c-b5fd-e9b02adba57d None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:02.997225 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28596344-7281-4bf3-adfd-39c8dd517dca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:02.998164 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28596344-7281-4bf3-adfd-39c8dd517dca None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:02.998513 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28596344-7281-4bf3-adfd-39c8dd517dca None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:02.998946 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28596344-7281-4bf3-adfd-39c8dd517dca None admin] PATH_INFO: `/v3/projects/1a4b3d97a66e4af4be0a066285f102de` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:03.000324 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28596344-7281-4bf3-adfd-39c8dd517dca None admin] RBAC: Authorizing `identity:get_project(project_id=1a4b3d97a66e4af4be0a066285f102de)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:03.002338 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28596344-7281-4bf3-adfd-39c8dd517dca None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:03.004266 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 58/125] 10.222.0.59 () {56 vars in 1186 bytes} [Tue Oct 11 11:46:02 2022] GET /identity/v3/projects/1a4b3d97a66e4af4be0a066285f102de => generated 298 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:46:03.012089 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28596344-7281-4bf3-adfd-39c8dd517dca None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:03.059143 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:03.131078 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fd7f3a-acbe-40d7-bddc-6ee9e37d6984 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:03.143690 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:03.168005 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28596344-7281-4bf3-adfd-39c8dd517dca None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:03.203463 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceb1dd1-065c-4e85-883f-876fbe5ab43d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:03.204523 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceb1dd1-065c-4e85-883f-876fbe5ab43d None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:03.205010 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceb1dd1-065c-4e85-883f-876fbe5ab43d None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:03.205503 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceb1dd1-065c-4e85-883f-876fbe5ab43d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:03.218232 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035a985c-2cfb-486d-b472-ab946b62e89d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:03.218821 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035a985c-2cfb-486d-b472-ab946b62e89d None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:03.219109 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035a985c-2cfb-486d-b472-ab946b62e89d None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:03.219388 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035a985c-2cfb-486d-b472-ab946b62e89d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:03.220343 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035a985c-2cfb-486d-b472-ab946b62e89d None admin] RBAC: Adding query filter params (role.id=3a1a5476791547419a0b885c2022ef68, scope.project.id=1a4b3d97a66e4af4be0a066285f102de, user.id=4965b07351a94e97bd57865e425127be) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:46:03.220672 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035a985c-2cfb-486d-b472-ab946b62e89d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:03.222243 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035a985c-2cfb-486d-b472-ab946b62e89d None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:03.229729 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 59/126] 10.222.0.59 () {56 vars in 1374 bytes} [Tue Oct 11 11:46:03 2022] GET /identity/v3/role_assignments?user.id=4965b07351a94e97bd57865e425127be&scope.project.id=1a4b3d97a66e4af4be0a066285f102de&role.id=3a1a5476791547419a0b885c2022ef68 => generated 608 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:46:03.323513 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ceb1dd1-065c-4e85-883f-876fbe5ab43d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:03.327012 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ceb1dd1-065c-4e85-883f-876fbe5ab43d None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:03.511948 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 68/127] 10.222.0.59 () {58 vars in 1282 bytes} [Tue Oct 11 11:46:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 636 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 11 11:46:03.517535 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035a985c-2cfb-486d-b472-ab946b62e89d None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:03.633843 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:03.655198 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035a985c-2cfb-486d-b472-ab946b62e89d None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:03.695977 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e295a3fd-ed00-46fb-b6e0-3f12295c2a82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:03.696894 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e295a3fd-ed00-46fb-b6e0-3f12295c2a82 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:03.697307 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e295a3fd-ed00-46fb-b6e0-3f12295c2a82 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:03.697703 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e295a3fd-ed00-46fb-b6e0-3f12295c2a82 None admin] PATH_INFO: `/v3/projects/1a4b3d97a66e4af4be0a066285f102de` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:03.699175 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e295a3fd-ed00-46fb-b6e0-3f12295c2a82 None admin] RBAC: Authorizing `identity:get_project(project_id=1a4b3d97a66e4af4be0a066285f102de)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:03.700809 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e295a3fd-ed00-46fb-b6e0-3f12295c2a82 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:03.708004 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 60/128] 10.222.0.59 () {56 vars in 1186 bytes} [Tue Oct 11 11:46:03 2022] GET /identity/v3/projects/1a4b3d97a66e4af4be0a066285f102de => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 11:46:03.709406 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceb1dd1-065c-4e85-883f-876fbe5ab43d None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:03.719553 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:03.839923 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceb1dd1-065c-4e85-883f-876fbe5ab43d None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:03.882645 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce28b044-f4c4-482d-8a1a-03b59b26354b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:03.883298 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce28b044-f4c4-482d-8a1a-03b59b26354b None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:03.883724 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce28b044-f4c4-482d-8a1a-03b59b26354b None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:03.883840 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce28b044-f4c4-482d-8a1a-03b59b26354b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:03.884578 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce28b044-f4c4-482d-8a1a-03b59b26354b None admin] RBAC: Adding query filter params (role.id=de2d2bd26fff4748a79990f97a107164, scope.project.id=1a4b3d97a66e4af4be0a066285f102de, user.id=4965b07351a94e97bd57865e425127be) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:46:03.884892 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce28b044-f4c4-482d-8a1a-03b59b26354b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:03.886450 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce28b044-f4c4-482d-8a1a-03b59b26354b None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:03.894004 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 69/129] 10.222.0.59 () {56 vars in 1374 bytes} [Tue Oct 11 11:46:03 2022] GET /identity/v3/role_assignments?user.id=4965b07351a94e97bd57865e425127be&scope.project.id=1a4b3d97a66e4af4be0a066285f102de&role.id=de2d2bd26fff4748a79990f97a107164 => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 11:46:04.842838 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9a9b41-954f-42c1-b3c0-7e612187ac3d None None] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:04.843412 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9a9b41-954f-42c1-b3c0-7e612187ac3d None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:04.843412 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9a9b41-954f-42c1-b3c0-7e612187ac3d None None] PATH_INFO: `/` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:04.844780 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 61/130] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:46:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:46:04.850509 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0200c4-9d85-44b8-baf7-702a3920b818 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:04.850916 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0200c4-9d85-44b8-baf7-702a3920b818 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:04.851266 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0200c4-9d85-44b8-baf7-702a3920b818 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:04.871106 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-0e0200c4-9d85-44b8-baf7-702a3920b818 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:46:04.872304 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-0e0200c4-9d85-44b8-baf7-702a3920b818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:46:04.943741 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:04.948871 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 70/131] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:46:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 11 11:46:05.264671 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9a9b41-954f-42c1-b3c0-7e612187ac3d None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:05.280198 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe51ed8-3247-497e-95c5-cadc35183faa None None] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:05.280481 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe51ed8-3247-497e-95c5-cadc35183faa None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:05.280760 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe51ed8-3247-497e-95c5-cadc35183faa None None] PATH_INFO: `/` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:05.282195 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 71/132] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:46:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 11 11:46:05.289372 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b59ee45-c195-4925-8f7c-ea7638a5e7cf None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:05.289777 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b59ee45-c195-4925-8f7c-ea7638a5e7cf None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:05.290191 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b59ee45-c195-4925-8f7c-ea7638a5e7cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:05.317714 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-4b59ee45-c195-4925-8f7c-ea7638a5e7cf None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:46:05.319556 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-4b59ee45-c195-4925-8f7c-ea7638a5e7cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:46:05.394294 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9a9b41-954f-42c1-b3c0-7e612187ac3d None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:05.411007 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 72/133] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:46:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 11 11:46:05.452201 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18774aa-a380-4a00-af8b-4b4300d0f9a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:05.453298 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18774aa-a380-4a00-af8b-4b4300d0f9a0 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:05.453936 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18774aa-a380-4a00-af8b-4b4300d0f9a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:05.454277 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18774aa-a380-4a00-af8b-4b4300d0f9a0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:05.473331 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18774aa-a380-4a00-af8b-4b4300d0f9a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 11:46:05.483959 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application [None req-e18774aa-a380-4a00-af8b-4b4300d0f9a0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 11 11:46:05.483959 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:46:05.483959 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:46:05.483959 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:46:05.483959 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:46:05.483959 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:46:05.483959 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:46:05.483959 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:46:05.483959 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:46:05.483959 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:46:05.483959 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:46:05.483959 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:46:05.483959 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 11:46:05.483959 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 11:46:05.483959 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 11:46:05.483959 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 11:46:05.485351 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 11:46:05.485351 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 11:46:05.485351 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 11:46:05.485351 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 11:46:05.485351 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:46:05.485351 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:46:05.485351 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:46:05.485351 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:46:05.485351 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:46:05.485351 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:46:05.485351 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:46:05.485351 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application with Lock( Oct 11 11:46:05.485351 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:46:05.485351 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:46:05.485351 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:46:05.485351 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:46:05.486519 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:46:05.486519 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:46:05.486519 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:46:05.486519 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:46:05.486519 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 11:46:05.486519 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 11:46:05.486519 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 11:46:05.486519 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 11:46:05.486519 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 11:46:05.486519 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 11:46:05.486519 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 11 11:46:05.486519 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:46:05.486519 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 62/134] 10.222.0.59 () {56 vars in 1099 bytes} [Tue Oct 11 11:46:05 2022] GET /identity/v3/roles/reader => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 11 11:46:05.495685 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b59ee45-c195-4925-8f7c-ea7638a5e7cf None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:05.620019 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b59ee45-c195-4925-8f7c-ea7638a5e7cf None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:05.659857 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:05.667739 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56cb5f8-c1b8-4c15-adc0-a6bae75e8099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:05.668486 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56cb5f8-c1b8-4c15-adc0-a6bae75e8099 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:05.668869 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56cb5f8-c1b8-4c15-adc0-a6bae75e8099 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:05.669192 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56cb5f8-c1b8-4c15-adc0-a6bae75e8099 None admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:05.669952 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56cb5f8-c1b8-4c15-adc0-a6bae75e8099 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:46:05.670307 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56cb5f8-c1b8-4c15-adc0-a6bae75e8099 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:05.671890 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56cb5f8-c1b8-4c15-adc0-a6bae75e8099 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:05.682494 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 73/135] 10.222.0.59 () {56 vars in 1101 bytes} [Tue Oct 11 11:46:05 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:46:05.688940 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18774aa-a380-4a00-af8b-4b4300d0f9a0 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:05.699841 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56cb5f8-c1b8-4c15-adc0-a6bae75e8099 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:05.710155 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:05.815785 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18774aa-a380-4a00-af8b-4b4300d0f9a0 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:05.853507 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56cb5f8-c1b8-4c15-adc0-a6bae75e8099 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:05.853919 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8daf1d-ca38-4a5c-a80a-dcc6bac0e2e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:05.854753 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8daf1d-ca38-4a5c-a80a-dcc6bac0e2e3 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:05.854898 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8daf1d-ca38-4a5c-a80a-dcc6bac0e2e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:05.855170 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8daf1d-ca38-4a5c-a80a-dcc6bac0e2e3 None admin] PATH_INFO: `/v3/users/dd79d21ab0b94e9f9d5b6baae3791a5e` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:05.856343 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8daf1d-ca38-4a5c-a80a-dcc6bac0e2e3 None admin] RBAC: Authorizing `identity:get_user(user_id=dd79d21ab0b94e9f9d5b6baae3791a5e)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:05.857811 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8daf1d-ca38-4a5c-a80a-dcc6bac0e2e3 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:05.859480 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 63/136] 10.222.0.59 () {56 vars in 1177 bytes} [Tue Oct 11 11:46:05 2022] GET /identity/v3/users/dd79d21ab0b94e9f9d5b6baae3791a5e => generated 289 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 11:46:05.866641 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8daf1d-ca38-4a5c-a80a-dcc6bac0e2e3 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:05.891896 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3a1fda-d78b-4246-9680-8b833cd047c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:05.892435 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3a1fda-d78b-4246-9680-8b833cd047c1 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:05.892711 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3a1fda-d78b-4246-9680-8b833cd047c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:05.892981 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3a1fda-d78b-4246-9680-8b833cd047c1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:05.894157 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3a1fda-d78b-4246-9680-8b833cd047c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:05.895656 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3a1fda-d78b-4246-9680-8b833cd047c1 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:05.897316 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 74/137] 10.222.0.59 () {56 vars in 1108 bytes} [Tue Oct 11 11:46:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 11:46:05.903426 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3a1fda-d78b-4246-9680-8b833cd047c1 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:05.961779 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:05.986405 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8daf1d-ca38-4a5c-a80a-dcc6bac0e2e3 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:05.987688 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:06.023149 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608e6aa2-b55f-42ff-9560-5073ed9fbc1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:06.023801 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608e6aa2-b55f-42ff-9560-5073ed9fbc1f None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:06.024071 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608e6aa2-b55f-42ff-9560-5073ed9fbc1f None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:06.024358 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608e6aa2-b55f-42ff-9560-5073ed9fbc1f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:06.054273 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3a1fda-d78b-4246-9680-8b833cd047c1 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:06.090780 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6124e03f-b240-4546-add4-a5769c855259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:06.091400 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6124e03f-b240-4546-add4-a5769c855259 None admin] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:06.091664 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6124e03f-b240-4546-add4-a5769c855259 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:06.091934 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6124e03f-b240-4546-add4-a5769c855259 None admin] PATH_INFO: `/v3/users` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:06.092586 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6124e03f-b240-4546-add4-a5769c855259 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:06.094041 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6124e03f-b240-4546-add4-a5769c855259 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:06.099183 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608e6aa2-b55f-42ff-9560-5073ed9fbc1f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:06.100681 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608e6aa2-b55f-42ff-9560-5073ed9fbc1f None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:06.117993 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 75/138] 10.222.0.59 () {60 vars in 1132 bytes} [Tue Oct 11 11:46:05 2022] POST /identity/v3/users => generated 283 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 11:46:06.198882 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:06.223156 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 64/139] 10.222.0.59 () {58 vars in 1282 bytes} [Tue Oct 11 11:46:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 11 11:46:06.229939 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6124e03f-b240-4546-add4-a5769c855259 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:06.361661 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6124e03f-b240-4546-add4-a5769c855259 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:06.384135 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:06.417701 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171c1e43-36fb-465a-af3f-055ad69daae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:06.418942 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171c1e43-36fb-465a-af3f-055ad69daae2 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:06.419428 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171c1e43-36fb-465a-af3f-055ad69daae2 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:06.419884 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171c1e43-36fb-465a-af3f-055ad69daae2 None admin] PATH_INFO: `/v3/projects/1a4b3d97a66e4af4be0a066285f102de` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:06.421666 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171c1e43-36fb-465a-af3f-055ad69daae2 None admin] RBAC: Authorizing `identity:get_project(project_id=1a4b3d97a66e4af4be0a066285f102de)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:06.424116 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171c1e43-36fb-465a-af3f-055ad69daae2 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:06.427431 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 76/140] 10.222.0.59 () {56 vars in 1186 bytes} [Tue Oct 11 11:46:06 2022] GET /identity/v3/projects/1a4b3d97a66e4af4be0a066285f102de => generated 298 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:46:06.433858 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608e6aa2-b55f-42ff-9560-5073ed9fbc1f None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:06.599317 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608e6aa2-b55f-42ff-9560-5073ed9fbc1f None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:06.660203 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69f8d35-2be8-41ea-ada7-8920d2b8464e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:06.661358 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69f8d35-2be8-41ea-ada7-8920d2b8464e None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:06.661846 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69f8d35-2be8-41ea-ada7-8920d2b8464e None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:06.662317 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69f8d35-2be8-41ea-ada7-8920d2b8464e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:06.663866 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69f8d35-2be8-41ea-ada7-8920d2b8464e None admin] RBAC: Adding query filter params (role.id=acda9dcd3a754265af99c2abc579d2da, scope.project.id=1a4b3d97a66e4af4be0a066285f102de, user.id=dd79d21ab0b94e9f9d5b6baae3791a5e) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:46:06.664490 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69f8d35-2be8-41ea-ada7-8920d2b8464e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:06.667330 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69f8d35-2be8-41ea-ada7-8920d2b8464e None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:06.679218 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 65/141] 10.222.0.59 () {56 vars in 1374 bytes} [Tue Oct 11 11:46:06 2022] GET /identity/v3/role_assignments?user.id=dd79d21ab0b94e9f9d5b6baae3791a5e&scope.project.id=1a4b3d97a66e4af4be0a066285f102de&role.id=acda9dcd3a754265af99c2abc579d2da => generated 608 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 11:46:08.434621 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4397483-bc6a-470b-9776-066b0fc99a0c None None] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:08.435200 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4397483-bc6a-470b-9776-066b0fc99a0c None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:08.435200 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4397483-bc6a-470b-9776-066b0fc99a0c None None] PATH_INFO: `/` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:08.436530 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 77/142] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:46:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:46:08.442119 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ee38c8-2369-4ebe-a62f-5be91b0e2403 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:08.442724 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ee38c8-2369-4ebe-a62f-5be91b0e2403 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:08.442724 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ee38c8-2369-4ebe-a62f-5be91b0e2403 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:08.464473 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-d8ee38c8-2369-4ebe-a62f-5be91b0e2403 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:46:08.465764 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-d8ee38c8-2369-4ebe-a62f-5be91b0e2403 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:46:08.516627 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:08.547115 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 66/143] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:46:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 11:46:08.809730 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4397483-bc6a-470b-9776-066b0fc99a0c None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:08.871781 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b35ed57-dd31-45f3-9d84-e91eb6d7830f None None] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:08.872449 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b35ed57-dd31-45f3-9d84-e91eb6d7830f None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:08.873421 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b35ed57-dd31-45f3-9d84-e91eb6d7830f None None] PATH_INFO: `/` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:08.876692 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 67/144] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:46:08 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:46:08.883460 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ab0bf5-1cdf-4720-9bf2-789885e1cb90 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:08.883743 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ab0bf5-1cdf-4720-9bf2-789885e1cb90 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:08.884016 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ab0bf5-1cdf-4720-9bf2-789885e1cb90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:08.903271 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-d9ab0bf5-1cdf-4720-9bf2-789885e1cb90 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:46:08.904489 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-d9ab0bf5-1cdf-4720-9bf2-789885e1cb90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:46:08.937300 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4397483-bc6a-470b-9776-066b0fc99a0c None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:08.979747 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ced62cd-0a98-4e49-a257-a50c49f0295d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:08.980413 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ced62cd-0a98-4e49-a257-a50c49f0295d None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:08.980712 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ced62cd-0a98-4e49-a257-a50c49f0295d None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:08.980991 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ced62cd-0a98-4e49-a257-a50c49f0295d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:08.994948 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ced62cd-0a98-4e49-a257-a50c49f0295d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 11:46:08.998630 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 68/145] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:46:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 11 11:46:09.000135 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application [None req-6ced62cd-0a98-4e49-a257-a50c49f0295d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 11 11:46:09.000135 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:46:09.000135 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:46:09.000135 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:46:09.000135 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:46:09.000135 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:46:09.000135 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:46:09.000135 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:46:09.000135 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:46:09.000135 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:46:09.000135 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:46:09.000135 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:46:09.000135 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 11:46:09.000135 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 11:46:09.000135 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 11:46:09.000135 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 11:46:09.001347 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 11:46:09.001347 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 11:46:09.001347 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 11:46:09.001347 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 11:46:09.001347 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:46:09.001347 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:46:09.001347 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:46:09.001347 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:46:09.001347 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:46:09.001347 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:46:09.001347 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:46:09.001347 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application with Lock( Oct 11 11:46:09.001347 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:46:09.001347 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:46:09.001347 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:46:09.001347 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:46:09.002185 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:46:09.002185 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:46:09.002185 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:46:09.002185 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:46:09.002185 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 11:46:09.002185 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 11:46:09.002185 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 11:46:09.002185 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 11:46:09.002185 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 11:46:09.002185 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 11:46:09.002185 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 11 11:46:09.002185 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:46:09.002185 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 78/146] 10.222.0.59 () {56 vars in 1096 bytes} [Tue Oct 11 11:46:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:46:09.010593 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ab0bf5-1cdf-4720-9bf2-789885e1cb90 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:09.145371 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ab0bf5-1cdf-4720-9bf2-789885e1cb90 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:09.148826 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:09.194531 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a8b937-3063-4bf2-962e-c52f07ad1de4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:09.194895 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a8b937-3063-4bf2-962e-c52f07ad1de4 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:09.195381 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a8b937-3063-4bf2-962e-c52f07ad1de4 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:09.195636 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a8b937-3063-4bf2-962e-c52f07ad1de4 None admin] PATH_INFO: `/v3/roles` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:09.196561 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a8b937-3063-4bf2-962e-c52f07ad1de4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:46:09.196561 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a8b937-3063-4bf2-962e-c52f07ad1de4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:09.197866 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a8b937-3063-4bf2-962e-c52f07ad1de4 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:09.208023 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 69/147] 10.222.0.59 () {56 vars in 1099 bytes} [Tue Oct 11 11:46:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:46:09.213615 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ced62cd-0a98-4e49-a257-a50c49f0295d None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:09.264017 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a8b937-3063-4bf2-962e-c52f07ad1de4 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:09.267118 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:09.356743 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ced62cd-0a98-4e49-a257-a50c49f0295d None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:09.390531 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a8b937-3063-4bf2-962e-c52f07ad1de4 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:09.418266 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5e60d6-ed6e-4e01-ab4a-64a2d3cdba13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:09.419314 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5e60d6-ed6e-4e01-ab4a-64a2d3cdba13 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:09.419821 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5e60d6-ed6e-4e01-ab4a-64a2d3cdba13 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:09.420301 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5e60d6-ed6e-4e01-ab4a-64a2d3cdba13 None admin] PATH_INFO: `/v3/users/1deb067c84ed458894cffcc0eba4330f` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:09.439375 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b052a74-e29e-4651-b469-83f030777250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:09.441128 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b052a74-e29e-4651-b469-83f030777250 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:09.441926 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b052a74-e29e-4651-b469-83f030777250 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:09.442812 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b052a74-e29e-4651-b469-83f030777250 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:09.446856 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b052a74-e29e-4651-b469-83f030777250 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:09.447462 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5e60d6-ed6e-4e01-ab4a-64a2d3cdba13 None admin] RBAC: Authorizing `identity:get_user(user_id=1deb067c84ed458894cffcc0eba4330f)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:09.449866 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5e60d6-ed6e-4e01-ab4a-64a2d3cdba13 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:09.451512 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b052a74-e29e-4651-b469-83f030777250 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:09.452551 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 79/148] 10.222.0.59 () {56 vars in 1177 bytes} [Tue Oct 11 11:46:09 2022] GET /identity/v3/users/1deb067c84ed458894cffcc0eba4330f => generated 283 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:46:09.461195 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5e60d6-ed6e-4e01-ab4a-64a2d3cdba13 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:09.466640 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 70/149] 10.222.0.59 () {56 vars in 1108 bytes} [Tue Oct 11 11:46:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:46:09.469704 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b052a74-e29e-4651-b469-83f030777250 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:09.522192 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:09.608444 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b052a74-e29e-4651-b469-83f030777250 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:09.612080 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5e60d6-ed6e-4e01-ab4a-64a2d3cdba13 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:09.613820 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:09.654893 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573ab00f-494f-47b3-b003-f4a5556af418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:09.655641 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573ab00f-494f-47b3-b003-f4a5556af418 None admin] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:09.655921 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573ab00f-494f-47b3-b003-f4a5556af418 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:09.656192 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573ab00f-494f-47b3-b003-f4a5556af418 None admin] PATH_INFO: `/v3/users` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:09.656872 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573ab00f-494f-47b3-b003-f4a5556af418 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:09.658345 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573ab00f-494f-47b3-b003-f4a5556af418 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:09.661097 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcd8439-aaa4-4a67-b516-9efc36e4a7b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:09.661767 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcd8439-aaa4-4a67-b516-9efc36e4a7b0 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:09.662076 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcd8439-aaa4-4a67-b516-9efc36e4a7b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:09.662311 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcd8439-aaa4-4a67-b516-9efc36e4a7b0 None admin] PATH_INFO: `/v3/projects/ed233e6a85f94375aece3faac28df210` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:09.663622 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcd8439-aaa4-4a67-b516-9efc36e4a7b0 None admin] RBAC: Authorizing `identity:get_project(project_id=ed233e6a85f94375aece3faac28df210)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:09.665169 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcd8439-aaa4-4a67-b516-9efc36e4a7b0 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:09.667032 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 80/150] 10.222.0.59 () {56 vars in 1186 bytes} [Tue Oct 11 11:46:09 2022] GET /identity/v3/projects/ed233e6a85f94375aece3faac28df210 => generated 302 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:46:09.676435 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcd8439-aaa4-4a67-b516-9efc36e4a7b0 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:09.705121 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 71/151] 10.222.0.59 () {60 vars in 1132 bytes} [Tue Oct 11 11:46:09 2022] POST /identity/v3/users => generated 297 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 11:46:09.803305 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529ec3ae-7319-413a-b839-2f1c9409b94c None None] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:09.803597 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529ec3ae-7319-413a-b839-2f1c9409b94c None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:09.803917 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529ec3ae-7319-413a-b839-2f1c9409b94c None None] PATH_INFO: `/` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:09.805502 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 72/152] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:46:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 11 11:46:09.811524 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b800fc-a5d6-4a4e-a961-013032177c42 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:09.811844 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b800fc-a5d6-4a4e-a961-013032177c42 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:09.812160 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b800fc-a5d6-4a4e-a961-013032177c42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:09.826850 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcd8439-aaa4-4a67-b516-9efc36e4a7b0 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:09.832721 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-d0b800fc-a5d6-4a4e-a961-013032177c42 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:46:09.834013 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-d0b800fc-a5d6-4a4e-a961-013032177c42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:46:09.870841 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400ba359-f11d-44c1-9474-78bb1f605251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:09.871565 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400ba359-f11d-44c1-9474-78bb1f605251 None admin] REQUEST_METHOD: `PUT` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:09.871871 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400ba359-f11d-44c1-9474-78bb1f605251 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:09.872113 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400ba359-f11d-44c1-9474-78bb1f605251 None admin] PATH_INFO: `/v3/projects/ed233e6a85f94375aece3faac28df210/users/1deb067c84ed458894cffcc0eba4330f/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:09.874296 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400ba359-f11d-44c1-9474-78bb1f605251 None admin] RBAC: Authorizing `identity:create_grant(project_id=ed233e6a85f94375aece3faac28df210, user_id=1deb067c84ed458894cffcc0eba4330f, role_id=5cf1f4bd2fef46ab9c6eff56d52f9f69)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:09.875870 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400ba359-f11d-44c1-9474-78bb1f605251 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:09.892833 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 81/153] 10.222.0.59 () {58 vars in 1439 bytes} [Tue Oct 11 11:46:09 2022] PUT /identity/v3/projects/ed233e6a85f94375aece3faac28df210/users/1deb067c84ed458894cffcc0eba4330f/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 11:46:09.934400 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 73/154] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:46:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 11 11:46:10.200701 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400ba359-f11d-44c1-9474-78bb1f605251 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:10.236521 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:10.353008 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400ba359-f11d-44c1-9474-78bb1f605251 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:10.415331 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff1c510-f47d-4a23-8140-15a2140b960c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:10.416371 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff1c510-f47d-4a23-8140-15a2140b960c None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:10.416834 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff1c510-f47d-4a23-8140-15a2140b960c None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:10.417325 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff1c510-f47d-4a23-8140-15a2140b960c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:10.437052 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff1c510-f47d-4a23-8140-15a2140b960c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 11:46:10.447870 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application [None req-6ff1c510-f47d-4a23-8140-15a2140b960c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 11 11:46:10.447870 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:46:10.447870 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:46:10.447870 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:46:10.447870 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:46:10.447870 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:46:10.447870 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:46:10.447870 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:46:10.447870 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:46:10.447870 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:46:10.447870 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:46:10.447870 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:46:10.447870 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 11:46:10.447870 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 11:46:10.447870 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 11:46:10.447870 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 11:46:10.449250 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 11:46:10.449250 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 11:46:10.449250 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 11:46:10.449250 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 11:46:10.449250 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:46:10.449250 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:46:10.449250 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:46:10.449250 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:46:10.449250 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:46:10.449250 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:46:10.449250 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:46:10.449250 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application with Lock( Oct 11 11:46:10.449250 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:46:10.449250 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:46:10.449250 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:46:10.449250 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:46:10.450033 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:46:10.450033 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:46:10.450033 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:46:10.450033 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:46:10.450033 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 11:46:10.450033 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 11:46:10.450033 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 11:46:10.450033 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 11:46:10.450033 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 11:46:10.450033 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 11:46:10.450033 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 11 11:46:10.450033 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:46:10.450033 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 82/155] 10.222.0.59 () {56 vars in 1114 bytes} [Tue Oct 11 11:46:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:46:10.460045 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b800fc-a5d6-4a4e-a961-013032177c42 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:10.498865 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:10.597041 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b800fc-a5d6-4a4e-a961-013032177c42 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:10.635199 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b139c111-653d-4742-b526-aff3c1cf024e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:10.635834 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b139c111-653d-4742-b526-aff3c1cf024e None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:10.636113 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b139c111-653d-4742-b526-aff3c1cf024e None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:10.636389 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b139c111-653d-4742-b526-aff3c1cf024e None admin] PATH_INFO: `/v3/roles` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:10.637062 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b139c111-653d-4742-b526-aff3c1cf024e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:46:10.637364 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b139c111-653d-4742-b526-aff3c1cf024e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:10.638813 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b139c111-653d-4742-b526-aff3c1cf024e None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:10.648356 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 74/156] 10.222.0.59 () {56 vars in 1111 bytes} [Tue Oct 11 11:46:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:46:10.654293 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff1c510-f47d-4a23-8140-15a2140b960c None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:10.803817 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:10.820769 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff1c510-f47d-4a23-8140-15a2140b960c None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:10.881660 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78eb71cd-2166-44a5-8869-183bc350c98d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:10.882775 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eb71cd-2166-44a5-8869-183bc350c98d None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:10.883238 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eb71cd-2166-44a5-8869-183bc350c98d None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:10.883701 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eb71cd-2166-44a5-8869-183bc350c98d None admin] PATH_INFO: `/v3/users/1deb067c84ed458894cffcc0eba4330f` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:10.885875 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78eb71cd-2166-44a5-8869-183bc350c98d None admin] RBAC: Authorizing `identity:get_user(user_id=1deb067c84ed458894cffcc0eba4330f)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:10.888653 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78eb71cd-2166-44a5-8869-183bc350c98d None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:10.891733 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 83/157] 10.222.0.59 () {56 vars in 1177 bytes} [Tue Oct 11 11:46:10 2022] GET /identity/v3/users/1deb067c84ed458894cffcc0eba4330f => generated 283 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 11:46:10.901406 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b139c111-653d-4742-b526-aff3c1cf024e None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:11.076270 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:11.115003 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b139c111-653d-4742-b526-aff3c1cf024e None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:11.183980 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d8d777-98fe-4acc-94df-03d5cbc92692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:11.185029 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d8d777-98fe-4acc-94df-03d5cbc92692 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:11.185510 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d8d777-98fe-4acc-94df-03d5cbc92692 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:11.185987 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d8d777-98fe-4acc-94df-03d5cbc92692 None admin] PATH_INFO: `/v3/projects/ed233e6a85f94375aece3faac28df210` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:11.188028 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d8d777-98fe-4acc-94df-03d5cbc92692 None admin] RBAC: Authorizing `identity:get_project(project_id=ed233e6a85f94375aece3faac28df210)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:11.190763 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d8d777-98fe-4acc-94df-03d5cbc92692 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:11.193948 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 75/158] 10.222.0.59 () {56 vars in 1186 bytes} [Tue Oct 11 11:46:10 2022] GET /identity/v3/projects/ed233e6a85f94375aece3faac28df210 => generated 302 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:46:11.204473 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78eb71cd-2166-44a5-8869-183bc350c98d None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:11.389366 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78eb71cd-2166-44a5-8869-183bc350c98d None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:11.455816 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e794444c-1551-4b3a-ad79-b64be83846ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:11.457010 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e794444c-1551-4b3a-ad79-b64be83846ec None admin] REQUEST_METHOD: `PUT` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:11.457509 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e794444c-1551-4b3a-ad79-b64be83846ec None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:11.458008 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e794444c-1551-4b3a-ad79-b64be83846ec None admin] PATH_INFO: `/v3/projects/ed233e6a85f94375aece3faac28df210/users/1deb067c84ed458894cffcc0eba4330f/roles/3a1a5476791547419a0b885c2022ef68` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:11.462014 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e794444c-1551-4b3a-ad79-b64be83846ec None admin] RBAC: Authorizing `identity:create_grant(project_id=ed233e6a85f94375aece3faac28df210, user_id=1deb067c84ed458894cffcc0eba4330f, role_id=3a1a5476791547419a0b885c2022ef68)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:11.464864 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e794444c-1551-4b3a-ad79-b64be83846ec None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:11.489573 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 84/159] 10.222.0.59 () {58 vars in 1439 bytes} [Tue Oct 11 11:46:11 2022] PUT /identity/v3/projects/ed233e6a85f94375aece3faac28df210/users/1deb067c84ed458894cffcc0eba4330f/roles/3a1a5476791547419a0b885c2022ef68 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 11:46:12.354945 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f6e89a-5fcc-48cf-9341-775a1c9b15dc None None] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:12.355515 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f6e89a-5fcc-48cf-9341-775a1c9b15dc None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:12.355515 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f6e89a-5fcc-48cf-9341-775a1c9b15dc None None] PATH_INFO: `/` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:12.357170 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 76/160] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:46:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:46:12.362852 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2108c88-2153-4fff-b65e-a3df796db4cf None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:12.363137 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2108c88-2153-4fff-b65e-a3df796db4cf None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:12.363404 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2108c88-2153-4fff-b65e-a3df796db4cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:12.376495 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771899f8-601d-4f31-86b6-6b35ea87420d None None] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:12.376822 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771899f8-601d-4f31-86b6-6b35ea87420d None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:12.377041 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771899f8-601d-4f31-86b6-6b35ea87420d None None] PATH_INFO: `/` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:12.378766 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 77/161] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:46:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 11 11:46:12.384976 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed176c12-258c-4e96-9ad2-83f4ee4b3a7b None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:12.385248 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed176c12-258c-4e96-9ad2-83f4ee4b3a7b None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:12.385517 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed176c12-258c-4e96-9ad2-83f4ee4b3a7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:12.387666 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-c2108c88-2153-4fff-b65e-a3df796db4cf None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:46:12.389410 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-c2108c88-2153-4fff-b65e-a3df796db4cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:46:12.405000 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:12.407342 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-ed176c12-258c-4e96-9ad2-83f4ee4b3a7b None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:46:12.410360 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-ed176c12-258c-4e96-9ad2-83f4ee4b3a7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:46:12.472899 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 85/162] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:46:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 11:46:12.537465 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 78/163] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:46:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 11 11:46:12.746811 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2108c88-2153-4fff-b65e-a3df796db4cf None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:12.805538 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed176c12-258c-4e96-9ad2-83f4ee4b3a7b None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:12.856263 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2108c88-2153-4fff-b65e-a3df796db4cf None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:12.897974 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f2ced8-c0a1-47ee-b4a9-ed8c7cbee5a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:12.898993 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f2ced8-c0a1-47ee-b4a9-ed8c7cbee5a4 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:12.899445 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f2ced8-c0a1-47ee-b4a9-ed8c7cbee5a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:12.899885 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f2ced8-c0a1-47ee-b4a9-ed8c7cbee5a4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:12.913314 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f2ced8-c0a1-47ee-b4a9-ed8c7cbee5a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 11:46:12.919145 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application [None req-a8f2ced8-c0a1-47ee-b4a9-ed8c7cbee5a4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 11 11:46:12.919145 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:46:12.919145 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:46:12.919145 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:46:12.919145 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:46:12.919145 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:46:12.919145 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:46:12.919145 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:46:12.919145 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:46:12.919145 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:46:12.919145 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:46:12.919145 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:46:12.919145 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 11:46:12.919145 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 11:46:12.919145 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 11:46:12.919145 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 11:46:12.920139 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 11:46:12.920139 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 11:46:12.920139 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 11:46:12.920139 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 11:46:12.920139 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:46:12.920139 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:46:12.920139 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:46:12.920139 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:46:12.920139 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:46:12.920139 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:46:12.920139 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:46:12.920139 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application with Lock( Oct 11 11:46:12.920139 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:46:12.920139 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:46:12.920139 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:46:12.920139 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:46:12.920905 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:46:12.920905 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:46:12.920905 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:46:12.920905 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:46:12.920905 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 11:46:12.920905 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 11:46:12.920905 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 11:46:12.920905 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 11:46:12.920905 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 11:46:12.920905 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 11:46:12.920905 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 11 11:46:12.920905 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:46:12.928524 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 86/164] 10.222.0.59 () {56 vars in 1099 bytes} [Tue Oct 11 11:46:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:46:12.930509 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f2ced8-c0a1-47ee-b4a9-ed8c7cbee5a4 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:12.946608 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:13.003623 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:13.042209 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed176c12-258c-4e96-9ad2-83f4ee4b3a7b None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:13.091374 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111ae807-3966-47a1-abaa-cb71df7a7c9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:13.092585 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111ae807-3966-47a1-abaa-cb71df7a7c9c None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:13.092888 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111ae807-3966-47a1-abaa-cb71df7a7c9c None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:13.093140 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111ae807-3966-47a1-abaa-cb71df7a7c9c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:13.094911 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111ae807-3966-47a1-abaa-cb71df7a7c9c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:13.099108 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111ae807-3966-47a1-abaa-cb71df7a7c9c None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:13.102177 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 79/165] 10.222.0.59 () {56 vars in 1108 bytes} [Tue Oct 11 11:46:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 11:46:13.103214 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f2ced8-c0a1-47ee-b4a9-ed8c7cbee5a4 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:13.104268 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1309077-76e0-475d-a195-fd6c1924a1f6 None None] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:13.106231 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1309077-76e0-475d-a195-fd6c1924a1f6 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:13.106547 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1309077-76e0-475d-a195-fd6c1924a1f6 None None] PATH_INFO: `/` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:13.108252 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 80/166] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:46:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:46:13.113342 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1309077-76e0-475d-a195-fd6c1924a1f6 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:13.153644 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebd8fb8-db78-48fd-9f98-677fff52a1bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:13.154663 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebd8fb8-db78-48fd-9f98-677fff52a1bc None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:13.154941 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebd8fb8-db78-48fd-9f98-677fff52a1bc None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:13.155264 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebd8fb8-db78-48fd-9f98-677fff52a1bc None admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:13.155948 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebd8fb8-db78-48fd-9f98-677fff52a1bc None admin] RBAC: Adding query filter params (name=member) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:46:13.156257 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebd8fb8-db78-48fd-9f98-677fff52a1bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:13.157664 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebd8fb8-db78-48fd-9f98-677fff52a1bc None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:13.167865 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 87/167] 10.222.0.59 () {56 vars in 1101 bytes} [Tue Oct 11 11:46:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 11:46:13.169030 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f41087-cc48-4ab7-8d80-2e2208a61103 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:13.169314 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f41087-cc48-4ab7-8d80-2e2208a61103 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:13.169576 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f41087-cc48-4ab7-8d80-2e2208a61103 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:13.197388 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-25f41087-cc48-4ab7-8d80-2e2208a61103 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:46:13.199372 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-25f41087-cc48-4ab7-8d80-2e2208a61103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:46:13.241366 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1309077-76e0-475d-a195-fd6c1924a1f6 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:13.287377 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57420284-a8cd-4c97-a648-3f86143ef293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:13.287928 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57420284-a8cd-4c97-a648-3f86143ef293 None admin] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:13.288211 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57420284-a8cd-4c97-a648-3f86143ef293 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:13.288482 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57420284-a8cd-4c97-a648-3f86143ef293 None admin] PATH_INFO: `/v3/users` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:13.289168 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57420284-a8cd-4c97-a648-3f86143ef293 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:13.290697 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57420284-a8cd-4c97-a648-3f86143ef293 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:13.311293 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:13.315780 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 81/168] 10.222.0.59 () {60 vars in 1132 bytes} [Tue Oct 11 11:46:13 2022] POST /identity/v3/users => generated 297 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 11:46:13.324037 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57420284-a8cd-4c97-a648-3f86143ef293 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:13.331031 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 88/169] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:46:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 11:46:13.345536 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:13.482152 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57420284-a8cd-4c97-a648-3f86143ef293 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:13.534218 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a048c2f-42cf-4b4e-9c98-afbeb19a09bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:13.535770 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a048c2f-42cf-4b4e-9c98-afbeb19a09bc None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:13.536240 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a048c2f-42cf-4b4e-9c98-afbeb19a09bc None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:13.536695 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a048c2f-42cf-4b4e-9c98-afbeb19a09bc None admin] PATH_INFO: `/v3/users/e4f8a783cf9d4a408590b1f974846197` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:13.553690 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a048c2f-42cf-4b4e-9c98-afbeb19a09bc None admin] RBAC: Authorizing `identity:get_user(user_id=e4f8a783cf9d4a408590b1f974846197)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:13.555972 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a048c2f-42cf-4b4e-9c98-afbeb19a09bc None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:13.558083 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 82/170] 10.222.0.59 () {56 vars in 1177 bytes} [Tue Oct 11 11:46:13 2022] GET /identity/v3/users/e4f8a783cf9d4a408590b1f974846197 => generated 297 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:46:13.564152 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f41087-cc48-4ab7-8d80-2e2208a61103 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:13.656436 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a048c2f-42cf-4b4e-9c98-afbeb19a09bc None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:13.718900 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f41087-cc48-4ab7-8d80-2e2208a61103 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:13.775634 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:13.804171 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532749cd-8ada-40b1-9392-4fcd3baa9993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:13.805857 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532749cd-8ada-40b1-9392-4fcd3baa9993 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:13.806376 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532749cd-8ada-40b1-9392-4fcd3baa9993 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:13.806879 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532749cd-8ada-40b1-9392-4fcd3baa9993 None admin] PATH_INFO: `/v3/projects/ed233e6a85f94375aece3faac28df210` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:13.808665 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532749cd-8ada-40b1-9392-4fcd3baa9993 None admin] RBAC: Authorizing `identity:get_project(project_id=ed233e6a85f94375aece3faac28df210)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:13.811250 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532749cd-8ada-40b1-9392-4fcd3baa9993 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:13.814833 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 89/171] 10.222.0.59 () {56 vars in 1186 bytes} [Tue Oct 11 11:46:13 2022] GET /identity/v3/projects/ed233e6a85f94375aece3faac28df210 => generated 302 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:46:13.827228 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532749cd-8ada-40b1-9392-4fcd3baa9993 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:13.851324 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a048c2f-42cf-4b4e-9c98-afbeb19a09bc None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:13.924279 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6e9b76-7ef2-4838-ad7c-7a39e15ad933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:13.925901 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6e9b76-7ef2-4838-ad7c-7a39e15ad933 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:13.926697 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6e9b76-7ef2-4838-ad7c-7a39e15ad933 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:13.927444 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6e9b76-7ef2-4838-ad7c-7a39e15ad933 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:13.940749 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:13.949111 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6e9b76-7ef2-4838-ad7c-7a39e15ad933 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 11:46:13.961000 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application [None req-bd6e9b76-7ef2-4838-ad7c-7a39e15ad933 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 11 11:46:13.961000 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:46:13.961000 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:46:13.961000 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:46:13.961000 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:46:13.961000 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:46:13.961000 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:46:13.961000 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:46:13.961000 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:46:13.961000 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:46:13.961000 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:46:13.961000 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:46:13.961000 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 11:46:13.961000 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 11:46:13.961000 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 11:46:13.961000 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 11:46:13.962617 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 11:46:13.962617 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 11:46:13.962617 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 11:46:13.962617 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 11:46:13.962617 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:46:13.962617 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:46:13.962617 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:46:13.962617 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:46:13.962617 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:46:13.962617 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:46:13.962617 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:46:13.962617 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application with Lock( Oct 11 11:46:13.962617 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:46:13.962617 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:46:13.962617 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:46:13.962617 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:46:13.964150 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:46:13.964150 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:46:13.964150 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:46:13.964150 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:46:13.964150 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 11:46:13.964150 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 11:46:13.964150 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 11:46:13.964150 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 11:46:13.964150 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 11:46:13.964150 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 11:46:13.964150 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 11 11:46:13.964150 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:46:13.969692 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 83/172] 10.222.0.59 () {56 vars in 1096 bytes} [Tue Oct 11 11:46:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 11 11:46:13.980321 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6e9b76-7ef2-4838-ad7c-7a39e15ad933 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:14.027436 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532749cd-8ada-40b1-9392-4fcd3baa9993 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:14.108509 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f29e59-5c4c-44ff-96cf-ce95d02f1435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:14.110664 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f29e59-5c4c-44ff-96cf-ce95d02f1435 None admin] REQUEST_METHOD: `PUT` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:14.110664 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f29e59-5c4c-44ff-96cf-ce95d02f1435 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:14.110664 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f29e59-5c4c-44ff-96cf-ce95d02f1435 None admin] PATH_INFO: `/v3/projects/ed233e6a85f94375aece3faac28df210/users/e4f8a783cf9d4a408590b1f974846197/roles/de2d2bd26fff4748a79990f97a107164` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:14.115465 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f29e59-5c4c-44ff-96cf-ce95d02f1435 None admin] RBAC: Authorizing `identity:create_grant(project_id=ed233e6a85f94375aece3faac28df210, user_id=e4f8a783cf9d4a408590b1f974846197, role_id=de2d2bd26fff4748a79990f97a107164)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:14.118171 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f29e59-5c4c-44ff-96cf-ce95d02f1435 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:14.158344 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 90/173] 10.222.0.59 () {58 vars in 1439 bytes} [Tue Oct 11 11:46:13 2022] PUT /identity/v3/projects/ed233e6a85f94375aece3faac28df210/users/e4f8a783cf9d4a408590b1f974846197/roles/de2d2bd26fff4748a79990f97a107164 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 11:46:14.196895 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6e9b76-7ef2-4838-ad7c-7a39e15ad933 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:14.244371 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:14.264967 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f025e47-223d-470c-97fd-13f464975684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:14.265864 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f025e47-223d-470c-97fd-13f464975684 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:14.266275 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f025e47-223d-470c-97fd-13f464975684 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:14.266717 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f025e47-223d-470c-97fd-13f464975684 None admin] PATH_INFO: `/v3/roles` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:14.267874 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f025e47-223d-470c-97fd-13f464975684 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:46:14.268349 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f025e47-223d-470c-97fd-13f464975684 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:14.270688 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f025e47-223d-470c-97fd-13f464975684 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:14.287201 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 84/174] 10.222.0.59 () {56 vars in 1099 bytes} [Tue Oct 11 11:46:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:46:14.297571 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f29e59-5c4c-44ff-96cf-ce95d02f1435 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:14.390414 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d0c0cd-40fd-4867-9c82-7e605236fa10 None None] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:14.391392 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d0c0cd-40fd-4867-9c82-7e605236fa10 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:14.392004 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d0c0cd-40fd-4867-9c82-7e605236fa10 None None] PATH_INFO: `/` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:14.395330 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 85/175] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:46:14 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 11 11:46:14.405096 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b4a363-9405-40d2-80bd-b0d310baf904 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:14.405779 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b4a363-9405-40d2-80bd-b0d310baf904 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:14.406385 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b4a363-9405-40d2-80bd-b0d310baf904 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:14.464666 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-39b4a363-9405-40d2-80bd-b0d310baf904 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:46:14.467093 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-39b4a363-9405-40d2-80bd-b0d310baf904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:46:14.527994 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f29e59-5c4c-44ff-96cf-ce95d02f1435 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:14.607024 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:14.623974 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d4fb57-87b1-4856-b4f6-448b3e1d40fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:14.625340 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d4fb57-87b1-4856-b4f6-448b3e1d40fa None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:14.625842 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d4fb57-87b1-4856-b4f6-448b3e1d40fa None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:14.626449 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d4fb57-87b1-4856-b4f6-448b3e1d40fa None admin] PATH_INFO: `/v3/users/1deb067c84ed458894cffcc0eba4330f` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:14.634617 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d4fb57-87b1-4856-b4f6-448b3e1d40fa None admin] RBAC: Authorizing `identity:get_user(user_id=1deb067c84ed458894cffcc0eba4330f)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:14.634617 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d4fb57-87b1-4856-b4f6-448b3e1d40fa None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:14.636590 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 91/176] 10.222.0.59 () {56 vars in 1177 bytes} [Tue Oct 11 11:46:14 2022] GET /identity/v3/users/1deb067c84ed458894cffcc0eba4330f => generated 283 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:46:14.648592 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d4fb57-87b1-4856-b4f6-448b3e1d40fa None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:14.687450 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 86/177] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:46:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 11:46:14.938146 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d4fb57-87b1-4856-b4f6-448b3e1d40fa None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:15.015096 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10931818-3300-40b7-838d-48ed3a927c1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:15.016755 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10931818-3300-40b7-838d-48ed3a927c1f None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:15.017490 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10931818-3300-40b7-838d-48ed3a927c1f None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:15.018196 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10931818-3300-40b7-838d-48ed3a927c1f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:15.167580 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:15.181795 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10931818-3300-40b7-838d-48ed3a927c1f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:15.185031 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10931818-3300-40b7-838d-48ed3a927c1f None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:15.359923 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b4a363-9405-40d2-80bd-b0d310baf904 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:15.409999 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 92/178] 10.222.0.59 () {58 vars in 1282 bytes} [Tue Oct 11 11:46:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 762 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 11 11:46:15.418454 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10931818-3300-40b7-838d-48ed3a927c1f None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:15.584279 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b4a363-9405-40d2-80bd-b0d310baf904 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:15.614183 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:15.642342 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:15.658831 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10931818-3300-40b7-838d-48ed3a927c1f None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:15.687290 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac95de1-a47f-441e-8f0b-fb561474b78c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:15.687986 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac95de1-a47f-441e-8f0b-fb561474b78c None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:15.688800 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac95de1-a47f-441e-8f0b-fb561474b78c None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:15.688800 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac95de1-a47f-441e-8f0b-fb561474b78c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:15.714762 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac95de1-a47f-441e-8f0b-fb561474b78c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 11:46:15.722373 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application [None req-2ac95de1-a47f-441e-8f0b-fb561474b78c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 11 11:46:15.722373 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:46:15.722373 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:46:15.722373 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:46:15.722373 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:46:15.722373 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:46:15.722373 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:46:15.722373 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:46:15.722373 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:46:15.722373 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:46:15.722373 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:46:15.722373 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:46:15.722373 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 11:46:15.722373 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 11:46:15.722373 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 11:46:15.722373 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 11:46:15.724165 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 11:46:15.724165 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 11:46:15.724165 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 11:46:15.724165 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 11:46:15.724165 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:46:15.724165 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:46:15.724165 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:46:15.724165 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:46:15.724165 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:46:15.724165 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:46:15.724165 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:46:15.724165 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application with Lock( Oct 11 11:46:15.724165 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:46:15.724165 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:46:15.724165 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:46:15.724165 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:46:15.725704 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:46:15.725704 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:46:15.725704 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:46:15.725704 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:46:15.725704 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 11:46:15.725704 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 11:46:15.725704 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 11:46:15.725704 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 11:46:15.725704 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 11:46:15.725704 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 11:46:15.725704 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 11 11:46:15.725704 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:46:15.726823 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 87/179] 10.222.0.59 () {56 vars in 1114 bytes} [Tue Oct 11 11:46:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 367 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:46:15.730594 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446f44a3-a1e4-4b3d-a3e8-54a55ae29a71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:15.730594 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446f44a3-a1e4-4b3d-a3e8-54a55ae29a71 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:15.730594 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446f44a3-a1e4-4b3d-a3e8-54a55ae29a71 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:15.730594 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446f44a3-a1e4-4b3d-a3e8-54a55ae29a71 None admin] PATH_INFO: `/v3/projects/ed233e6a85f94375aece3faac28df210` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:15.731620 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446f44a3-a1e4-4b3d-a3e8-54a55ae29a71 None admin] RBAC: Authorizing `identity:get_project(project_id=ed233e6a85f94375aece3faac28df210)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:15.732394 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446f44a3-a1e4-4b3d-a3e8-54a55ae29a71 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:15.734609 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 93/180] 10.222.0.59 () {56 vars in 1186 bytes} [Tue Oct 11 11:46:15 2022] GET /identity/v3/projects/ed233e6a85f94375aece3faac28df210 => generated 302 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:46:15.745366 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac95de1-a47f-441e-8f0b-fb561474b78c None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:15.746949 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446f44a3-a1e4-4b3d-a3e8-54a55ae29a71 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:15.953395 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac95de1-a47f-441e-8f0b-fb561474b78c None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:16.032292 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446f44a3-a1e4-4b3d-a3e8-54a55ae29a71 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:16.075593 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcc0b09-83c7-44df-83ad-d0aa47cc90dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:16.076228 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcc0b09-83c7-44df-83ad-d0aa47cc90dd None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:16.076516 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcc0b09-83c7-44df-83ad-d0aa47cc90dd None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:16.076796 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcc0b09-83c7-44df-83ad-d0aa47cc90dd None admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:16.077502 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcc0b09-83c7-44df-83ad-d0aa47cc90dd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:46:16.077811 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcc0b09-83c7-44df-83ad-d0aa47cc90dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:16.079284 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcc0b09-83c7-44df-83ad-d0aa47cc90dd None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:16.083853 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb21bd9-527f-4496-ba5c-3a37e40a808e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:16.084947 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb21bd9-527f-4496-ba5c-3a37e40a808e None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:16.085441 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb21bd9-527f-4496-ba5c-3a37e40a808e None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:16.085937 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb21bd9-527f-4496-ba5c-3a37e40a808e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:16.087381 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb21bd9-527f-4496-ba5c-3a37e40a808e None admin] RBAC: Adding query filter params (role.id=5cf1f4bd2fef46ab9c6eff56d52f9f69, scope.project.id=ed233e6a85f94375aece3faac28df210, user.id=1deb067c84ed458894cffcc0eba4330f) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:46:16.087926 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb21bd9-527f-4496-ba5c-3a37e40a808e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:16.089847 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 94/181] 10.222.0.59 () {56 vars in 1111 bytes} [Tue Oct 11 11:46:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:46:16.090803 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb21bd9-527f-4496-ba5c-3a37e40a808e None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:16.099969 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcc0b09-83c7-44df-83ad-d0aa47cc90dd None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:16.103924 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 88/182] 10.222.0.59 () {56 vars in 1374 bytes} [Tue Oct 11 11:46:15 2022] GET /identity/v3/role_assignments?user.id=1deb067c84ed458894cffcc0eba4330f&scope.project.id=ed233e6a85f94375aece3faac28df210&role.id=5cf1f4bd2fef46ab9c6eff56d52f9f69 => generated 608 bytes in 360 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:46:16.185589 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:16.365411 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcc0b09-83c7-44df-83ad-d0aa47cc90dd None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:16.430540 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4651dc-fc34-4f23-b721-6146acfd453c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:16.432403 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4651dc-fc34-4f23-b721-6146acfd453c None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:16.433243 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4651dc-fc34-4f23-b721-6146acfd453c None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:16.434022 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4651dc-fc34-4f23-b721-6146acfd453c None admin] PATH_INFO: `/v3/users/1deb067c84ed458894cffcc0eba4330f` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:16.436727 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4651dc-fc34-4f23-b721-6146acfd453c None admin] RBAC: Authorizing `identity:get_user(user_id=1deb067c84ed458894cffcc0eba4330f)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:16.440238 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4651dc-fc34-4f23-b721-6146acfd453c None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:16.444112 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 95/183] 10.222.0.59 () {56 vars in 1177 bytes} [Tue Oct 11 11:46:16 2022] GET /identity/v3/users/1deb067c84ed458894cffcc0eba4330f => generated 283 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 11:46:16.453834 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb21bd9-527f-4496-ba5c-3a37e40a808e None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:16.557559 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:16.680135 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb21bd9-527f-4496-ba5c-3a37e40a808e None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:16.756403 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299721b0-0a92-4c07-89de-498a04323684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:16.757390 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299721b0-0a92-4c07-89de-498a04323684 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:16.757826 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299721b0-0a92-4c07-89de-498a04323684 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:16.758309 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299721b0-0a92-4c07-89de-498a04323684 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:16.900774 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299721b0-0a92-4c07-89de-498a04323684 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:16.903591 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299721b0-0a92-4c07-89de-498a04323684 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:16.978008 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:17.024647 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f09871c-c8c3-4581-ac47-8abdba6a1bbc None None] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:17.026241 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f09871c-c8c3-4581-ac47-8abdba6a1bbc None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:17.027004 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f09871c-c8c3-4581-ac47-8abdba6a1bbc None None] PATH_INFO: `/` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:17.029890 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 96/184] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:46:17 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 11 11:46:17.040273 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f67d2fc-0893-4192-873b-2409adeb753a None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:17.041128 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f67d2fc-0893-4192-873b-2409adeb753a None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:17.041926 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f67d2fc-0893-4192-873b-2409adeb753a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:17.075902 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-8f67d2fc-0893-4192-873b-2409adeb753a None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:46:17.077894 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-8f67d2fc-0893-4192-873b-2409adeb753a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:46:17.102054 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 89/185] 10.222.0.59 () {58 vars in 1282 bytes} [Tue Oct 11 11:46:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 650 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 11 11:46:17.111202 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299721b0-0a92-4c07-89de-498a04323684 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:17.205256 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 97/186] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:46:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 11 11:46:17.206798 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b9da44-f7ef-4047-aa77-e0a7500ac578 None None] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:17.207091 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b9da44-f7ef-4047-aa77-e0a7500ac578 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:17.207376 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b9da44-f7ef-4047-aa77-e0a7500ac578 None None] PATH_INFO: `/` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:17.208837 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 98/187] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:46:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:46:17.214602 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8db6510-934f-465b-bfba-e663e7edf200 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:17.214889 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8db6510-934f-465b-bfba-e663e7edf200 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:17.215193 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8db6510-934f-465b-bfba-e663e7edf200 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:17.242424 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-a8db6510-934f-465b-bfba-e663e7edf200 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:46:17.244404 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-a8db6510-934f-465b-bfba-e663e7edf200 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:46:17.256328 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299721b0-0a92-4c07-89de-498a04323684 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:17.283002 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:17.293862 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4414221-3222-47d8-a022-e6e462b83352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:17.295094 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4414221-3222-47d8-a022-e6e462b83352 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:17.295295 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4414221-3222-47d8-a022-e6e462b83352 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:17.295671 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4414221-3222-47d8-a022-e6e462b83352 None admin] PATH_INFO: `/v3/projects/ed233e6a85f94375aece3faac28df210` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:17.296991 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4414221-3222-47d8-a022-e6e462b83352 None admin] RBAC: Authorizing `identity:get_project(project_id=ed233e6a85f94375aece3faac28df210)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:17.298789 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4414221-3222-47d8-a022-e6e462b83352 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:17.305861 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 90/188] 10.222.0.59 () {56 vars in 1186 bytes} [Tue Oct 11 11:46:17 2022] GET /identity/v3/projects/ed233e6a85f94375aece3faac28df210 => generated 302 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:46:17.307432 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4414221-3222-47d8-a022-e6e462b83352 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:17.320639 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:17.335679 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 99/189] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:46:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 11 11:46:17.419655 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4414221-3222-47d8-a022-e6e462b83352 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:17.458454 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ace9102-1daf-44e7-99a0-d579b4fe4b54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:17.459383 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ace9102-1daf-44e7-99a0-d579b4fe4b54 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:17.459806 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ace9102-1daf-44e7-99a0-d579b4fe4b54 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:17.460243 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ace9102-1daf-44e7-99a0-d579b4fe4b54 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:17.461121 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ace9102-1daf-44e7-99a0-d579b4fe4b54 None admin] RBAC: Adding query filter params (role.id=3a1a5476791547419a0b885c2022ef68, scope.project.id=ed233e6a85f94375aece3faac28df210, user.id=1deb067c84ed458894cffcc0eba4330f) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:46:17.461572 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ace9102-1daf-44e7-99a0-d579b4fe4b54 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:17.463466 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ace9102-1daf-44e7-99a0-d579b4fe4b54 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:17.470778 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 91/190] 10.222.0.59 () {56 vars in 1374 bytes} [Tue Oct 11 11:46:17 2022] GET /identity/v3/role_assignments?user.id=1deb067c84ed458894cffcc0eba4330f&scope.project.id=ed233e6a85f94375aece3faac28df210&role.id=3a1a5476791547419a0b885c2022ef68 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:46:17.480087 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8db6510-934f-465b-bfba-e663e7edf200 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:17.621265 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8db6510-934f-465b-bfba-e663e7edf200 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:17.632554 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ace9102-1daf-44e7-99a0-d579b4fe4b54 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:17.677627 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ade1cbc-dac2-4d4a-920d-b770bb1c2ced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:17.678352 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ade1cbc-dac2-4d4a-920d-b770bb1c2ced None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:17.678683 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ade1cbc-dac2-4d4a-920d-b770bb1c2ced None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:17.678996 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ade1cbc-dac2-4d4a-920d-b770bb1c2ced None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:17.692528 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ade1cbc-dac2-4d4a-920d-b770bb1c2ced None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 11:46:17.697848 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application [None req-9ade1cbc-dac2-4d4a-920d-b770bb1c2ced None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 11 11:46:17.697848 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:46:17.697848 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:46:17.697848 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:46:17.697848 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:46:17.697848 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:46:17.697848 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:46:17.697848 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:46:17.697848 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:46:17.697848 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:46:17.697848 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:46:17.697848 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:46:17.697848 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 11:46:17.697848 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 11:46:17.697848 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 11:46:17.697848 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 11:46:17.699055 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 11:46:17.699055 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 11:46:17.699055 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 11:46:17.699055 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 11:46:17.699055 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:46:17.699055 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:46:17.699055 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:46:17.699055 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:46:17.699055 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:46:17.699055 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:46:17.699055 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:46:17.699055 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application with Lock( Oct 11 11:46:17.699055 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:46:17.699055 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:46:17.699055 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:46:17.699055 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:46:17.700047 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:46:17.700047 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:46:17.700047 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:46:17.700047 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:46:17.700047 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 11:46:17.700047 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 11:46:17.700047 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 11:46:17.700047 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 11:46:17.700047 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 11:46:17.700047 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 11:46:17.700047 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 11 11:46:17.700047 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:46:17.700047 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 100/191] 10.222.0.59 () {56 vars in 1099 bytes} [Tue Oct 11 11:46:17 2022] GET /identity/v3/roles/reader => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:46:17.707481 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ade1cbc-dac2-4d4a-920d-b770bb1c2ced None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:17.731062 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:17.763646 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:17.814154 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ace9102-1daf-44e7-99a0-d579b4fe4b54 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:17.866767 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfdc09a-d225-459e-9307-e0f4a6e3fd4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:17.867255 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ade1cbc-dac2-4d4a-920d-b770bb1c2ced None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:17.867444 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfdc09a-d225-459e-9307-e0f4a6e3fd4a None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:17.867711 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfdc09a-d225-459e-9307-e0f4a6e3fd4a None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:17.867993 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfdc09a-d225-459e-9307-e0f4a6e3fd4a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:17.869524 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfdc09a-d225-459e-9307-e0f4a6e3fd4a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:17.872045 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfdc09a-d225-459e-9307-e0f4a6e3fd4a None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:17.874008 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 92/192] 10.222.0.59 () {56 vars in 1108 bytes} [Tue Oct 11 11:46:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 11:46:17.879821 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfdc09a-d225-459e-9307-e0f4a6e3fd4a None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:17.910810 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86028b0-9c9e-4141-b3c2-2b1348fd56bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:17.911504 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86028b0-9c9e-4141-b3c2-2b1348fd56bd None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:17.911764 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86028b0-9c9e-4141-b3c2-2b1348fd56bd None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:17.912043 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86028b0-9c9e-4141-b3c2-2b1348fd56bd None admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:17.912751 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86028b0-9c9e-4141-b3c2-2b1348fd56bd None admin] RBAC: Adding query filter params (name=reader) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:46:17.913090 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86028b0-9c9e-4141-b3c2-2b1348fd56bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:17.914525 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86028b0-9c9e-4141-b3c2-2b1348fd56bd None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:17.929642 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 101/193] 10.222.0.59 () {56 vars in 1101 bytes} [Tue Oct 11 11:46:17 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 11:46:17.935541 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86028b0-9c9e-4141-b3c2-2b1348fd56bd None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:18.032015 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfdc09a-d225-459e-9307-e0f4a6e3fd4a None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:18.046749 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:18.075357 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86028b0-9c9e-4141-b3c2-2b1348fd56bd None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:18.083004 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc0d351-f8e6-4f73-b837-183154f422e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:18.084080 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc0d351-f8e6-4f73-b837-183154f422e2 None admin] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:18.084604 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc0d351-f8e6-4f73-b837-183154f422e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:18.085063 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc0d351-f8e6-4f73-b837-183154f422e2 None admin] PATH_INFO: `/v3/users` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:18.086345 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc0d351-f8e6-4f73-b837-183154f422e2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:18.089310 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc0d351-f8e6-4f73-b837-183154f422e2 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:18.106189 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:18.116629 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a98fd9-49d3-45a5-8673-b21022247cbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:18.117303 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a98fd9-49d3-45a5-8673-b21022247cbe None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:18.117582 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a98fd9-49d3-45a5-8673-b21022247cbe None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:18.117862 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a98fd9-49d3-45a5-8673-b21022247cbe None admin] PATH_INFO: `/v3/users/e982a82472774047842211f743f936d9` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:18.132260 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 93/194] 10.222.0.59 () {60 vars in 1132 bytes} [Tue Oct 11 11:46:17 2022] POST /identity/v3/users => generated 293 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 11:46:18.132260 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c7fe6f-9dbd-49f5-a3d5-d33af543d322 None None] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:18.132260 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c7fe6f-9dbd-49f5-a3d5-d33af543d322 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:18.132260 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c7fe6f-9dbd-49f5-a3d5-d33af543d322 None None] PATH_INFO: `/` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:18.132260 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 94/195] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:46:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:46:18.138616 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8dc946-be2c-4766-8299-4a571fc51cab None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:18.139135 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8dc946-be2c-4766-8299-4a571fc51cab None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:18.139563 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8dc946-be2c-4766-8299-4a571fc51cab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:18.143215 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a98fd9-49d3-45a5-8673-b21022247cbe None admin] RBAC: Authorizing `identity:get_user(user_id=e982a82472774047842211f743f936d9)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:18.145224 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a98fd9-49d3-45a5-8673-b21022247cbe None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:18.147275 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 102/196] 10.222.0.59 () {56 vars in 1177 bytes} [Tue Oct 11 11:46:17 2022] GET /identity/v3/users/e982a82472774047842211f743f936d9 => generated 297 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 11:46:18.156255 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a98fd9-49d3-45a5-8673-b21022247cbe None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:18.168632 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-9e8dc946-be2c-4766-8299-4a571fc51cab None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:46:18.170483 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-9e8dc946-be2c-4766-8299-4a571fc51cab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:46:18.261867 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 95/197] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:46:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 11 11:46:18.312609 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a98fd9-49d3-45a5-8673-b21022247cbe None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:18.358203 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f81715d-1a57-457d-ad6d-8f38aa5372b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:18.359416 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f81715d-1a57-457d-ad6d-8f38aa5372b4 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:18.359941 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f81715d-1a57-457d-ad6d-8f38aa5372b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:18.360513 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f81715d-1a57-457d-ad6d-8f38aa5372b4 None admin] PATH_INFO: `/v3/projects/ed233e6a85f94375aece3faac28df210` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:18.362243 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f81715d-1a57-457d-ad6d-8f38aa5372b4 None admin] RBAC: Authorizing `identity:get_project(project_id=ed233e6a85f94375aece3faac28df210)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:18.364295 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f81715d-1a57-457d-ad6d-8f38aa5372b4 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:18.366609 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 103/198] 10.222.0.59 () {56 vars in 1186 bytes} [Tue Oct 11 11:46:18 2022] GET /identity/v3/projects/ed233e6a85f94375aece3faac28df210 => generated 302 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 11:46:18.373556 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8dc946-be2c-4766-8299-4a571fc51cab None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:18.525242 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8dc946-be2c-4766-8299-4a571fc51cab None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:18.529254 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f81715d-1a57-457d-ad6d-8f38aa5372b4 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:18.532592 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:18.566205 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:18.582778 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a189e473-4805-49f8-a71d-ba77571ac8d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:18.583463 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a189e473-4805-49f8-a71d-ba77571ac8d4 None admin] REQUEST_METHOD: `PUT` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:18.583764 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a189e473-4805-49f8-a71d-ba77571ac8d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:18.584031 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a189e473-4805-49f8-a71d-ba77571ac8d4 None admin] PATH_INFO: `/v3/projects/ed233e6a85f94375aece3faac28df210/users/e982a82472774047842211f743f936d9/roles/acda9dcd3a754265af99c2abc579d2da` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:18.586117 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a189e473-4805-49f8-a71d-ba77571ac8d4 None admin] RBAC: Authorizing `identity:create_grant(project_id=ed233e6a85f94375aece3faac28df210, user_id=e982a82472774047842211f743f936d9, role_id=acda9dcd3a754265af99c2abc579d2da)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:18.587646 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a189e473-4805-49f8-a71d-ba77571ac8d4 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:18.602964 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 96/199] 10.222.0.59 () {58 vars in 1439 bytes} [Tue Oct 11 11:46:18 2022] PUT /identity/v3/projects/ed233e6a85f94375aece3faac28df210/users/e982a82472774047842211f743f936d9/roles/acda9dcd3a754265af99c2abc579d2da => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 11:46:18.745003 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f81715d-1a57-457d-ad6d-8f38aa5372b4 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:18.818022 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fff9c9-bb75-4608-82ce-de61443457c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:18.819135 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fff9c9-bb75-4608-82ce-de61443457c1 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:18.819605 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fff9c9-bb75-4608-82ce-de61443457c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:18.820074 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fff9c9-bb75-4608-82ce-de61443457c1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:18.839919 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fff9c9-bb75-4608-82ce-de61443457c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 11:46:18.850460 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application [None req-60fff9c9-bb75-4608-82ce-de61443457c1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 11 11:46:18.850460 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:46:18.850460 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:46:18.850460 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:46:18.850460 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:46:18.850460 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:46:18.850460 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:46:18.850460 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:46:18.850460 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:46:18.850460 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:46:18.850460 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:46:18.850460 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:46:18.850460 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 11:46:18.850460 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 11:46:18.850460 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 11:46:18.850460 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 11:46:18.852134 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 11:46:18.852134 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 11:46:18.852134 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 11:46:18.852134 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 11:46:18.852134 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:46:18.852134 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:46:18.852134 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:46:18.852134 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:46:18.852134 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:46:18.852134 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:46:18.852134 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:46:18.852134 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application with Lock( Oct 11 11:46:18.852134 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:46:18.852134 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:46:18.852134 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:46:18.852134 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:46:18.854489 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:46:18.854489 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:46:18.854489 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:46:18.854489 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:46:18.854489 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 11:46:18.854489 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 11:46:18.854489 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 11:46:18.854489 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 11:46:18.854489 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 11:46:18.854489 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 11:46:18.854489 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 11 11:46:18.854489 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:46:18.854489 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 104/200] 10.222.0.59 () {56 vars in 1099 bytes} [Tue Oct 11 11:46:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 11 11:46:18.863772 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a189e473-4805-49f8-a71d-ba77571ac8d4 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:18.983816 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a189e473-4805-49f8-a71d-ba77571ac8d4 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:19.025279 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2cc5b2-973c-4534-a569-2c496c80bb99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:19.025958 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2cc5b2-973c-4534-a569-2c496c80bb99 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:19.026239 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2cc5b2-973c-4534-a569-2c496c80bb99 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:19.026510 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2cc5b2-973c-4534-a569-2c496c80bb99 None admin] PATH_INFO: `/v3/roles` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:19.027241 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2cc5b2-973c-4534-a569-2c496c80bb99 None admin] RBAC: Adding query filter params (name=member) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:46:19.027544 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2cc5b2-973c-4534-a569-2c496c80bb99 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:19.028983 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2cc5b2-973c-4534-a569-2c496c80bb99 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:19.040125 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 97/201] 10.222.0.59 () {56 vars in 1101 bytes} [Tue Oct 11 11:46:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:46:19.046686 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fff9c9-bb75-4608-82ce-de61443457c1 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:19.161310 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:19.214737 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fff9c9-bb75-4608-82ce-de61443457c1 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:19.251146 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92804489-04eb-4547-b96b-f5cd98d9a85a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:19.251816 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92804489-04eb-4547-b96b-f5cd98d9a85a None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:19.252094 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92804489-04eb-4547-b96b-f5cd98d9a85a None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:19.252367 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92804489-04eb-4547-b96b-f5cd98d9a85a None admin] PATH_INFO: `/v3/users/e4f8a783cf9d4a408590b1f974846197` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:19.253626 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92804489-04eb-4547-b96b-f5cd98d9a85a None admin] RBAC: Authorizing `identity:get_user(user_id=e4f8a783cf9d4a408590b1f974846197)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:19.255185 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92804489-04eb-4547-b96b-f5cd98d9a85a None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:19.257194 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 105/202] 10.222.0.59 () {56 vars in 1177 bytes} [Tue Oct 11 11:46:19 2022] GET /identity/v3/users/e4f8a783cf9d4a408590b1f974846197 => generated 297 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:46:19.263028 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2cc5b2-973c-4534-a569-2c496c80bb99 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:19.316620 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:19.383970 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2cc5b2-973c-4534-a569-2c496c80bb99 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:19.424282 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405cc2d7-40a0-43b9-8b16-bacf1d9168a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:19.424916 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405cc2d7-40a0-43b9-8b16-bacf1d9168a6 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:19.425200 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405cc2d7-40a0-43b9-8b16-bacf1d9168a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:19.425482 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405cc2d7-40a0-43b9-8b16-bacf1d9168a6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:19.506720 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405cc2d7-40a0-43b9-8b16-bacf1d9168a6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:19.508450 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405cc2d7-40a0-43b9-8b16-bacf1d9168a6 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:19.559145 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:19.616676 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 98/203] 10.222.0.59 () {58 vars in 1282 bytes} [Tue Oct 11 11:46:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 11 11:46:19.622496 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92804489-04eb-4547-b96b-f5cd98d9a85a None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:19.738584 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92804489-04eb-4547-b96b-f5cd98d9a85a None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:19.774458 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3b6d1a-8209-4049-bd03-e1f35500379f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:19.775150 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3b6d1a-8209-4049-bd03-e1f35500379f None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:19.775435 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3b6d1a-8209-4049-bd03-e1f35500379f None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:19.775711 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3b6d1a-8209-4049-bd03-e1f35500379f None admin] PATH_INFO: `/v3/projects/ed233e6a85f94375aece3faac28df210` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:19.776866 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3b6d1a-8209-4049-bd03-e1f35500379f None admin] RBAC: Authorizing `identity:get_project(project_id=ed233e6a85f94375aece3faac28df210)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:19.778472 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3b6d1a-8209-4049-bd03-e1f35500379f None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:19.780754 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:19.781045 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 106/204] 10.222.0.59 () {56 vars in 1186 bytes} [Tue Oct 11 11:46:19 2022] GET /identity/v3/projects/ed233e6a85f94375aece3faac28df210 => generated 302 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 11:46:19.787148 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405cc2d7-40a0-43b9-8b16-bacf1d9168a6 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:19.919251 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405cc2d7-40a0-43b9-8b16-bacf1d9168a6 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:19.980433 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa4f6ba-d7eb-4bf8-acef-aefd3b3bbc25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:19.981447 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa4f6ba-d7eb-4bf8-acef-aefd3b3bbc25 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:19.981928 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa4f6ba-d7eb-4bf8-acef-aefd3b3bbc25 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:19.982404 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa4f6ba-d7eb-4bf8-acef-aefd3b3bbc25 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:19.983799 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa4f6ba-d7eb-4bf8-acef-aefd3b3bbc25 None admin] RBAC: Adding query filter params (role.id=de2d2bd26fff4748a79990f97a107164, scope.project.id=ed233e6a85f94375aece3faac28df210, user.id=e4f8a783cf9d4a408590b1f974846197) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:46:19.984337 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa4f6ba-d7eb-4bf8-acef-aefd3b3bbc25 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:19.986990 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa4f6ba-d7eb-4bf8-acef-aefd3b3bbc25 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:20.003147 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 99/205] 10.222.0.59 () {56 vars in 1374 bytes} [Tue Oct 11 11:46:19 2022] GET /identity/v3/role_assignments?user.id=e4f8a783cf9d4a408590b1f974846197&scope.project.id=ed233e6a85f94375aece3faac28df210&role.id=de2d2bd26fff4748a79990f97a107164 => generated 608 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:46:21.233235 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddcdb1b-dfa8-4d4b-8496-f1633d9f14d9 None None] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:21.233931 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddcdb1b-dfa8-4d4b-8496-f1633d9f14d9 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:21.234265 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddcdb1b-dfa8-4d4b-8496-f1633d9f14d9 None None] PATH_INFO: `/` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:21.236914 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 107/206] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:46:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:46:21.246522 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793d5e74-e4c7-45d3-93d1-a87de3a893c2 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:21.247240 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793d5e74-e4c7-45d3-93d1-a87de3a893c2 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:21.247532 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793d5e74-e4c7-45d3-93d1-a87de3a893c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:21.280187 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-793d5e74-e4c7-45d3-93d1-a87de3a893c2 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:46:21.282213 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-793d5e74-e4c7-45d3-93d1-a87de3a893c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:46:21.289211 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:21.332146 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cedbfbc-66b6-480c-9ee6-5d151a0a52ac None None] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:21.332800 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cedbfbc-66b6-480c-9ee6-5d151a0a52ac None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:21.333394 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cedbfbc-66b6-480c-9ee6-5d151a0a52ac None None] PATH_INFO: `/` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:21.336041 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 108/207] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:46:21 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:46:21.344388 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27f517c-b2ae-4b7d-a1a6-d885b1be5fc8 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:21.344863 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27f517c-b2ae-4b7d-a1a6-d885b1be5fc8 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:21.345373 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27f517c-b2ae-4b7d-a1a6-d885b1be5fc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:21.381930 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-c27f517c-b2ae-4b7d-a1a6-d885b1be5fc8 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:46:21.386200 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-c27f517c-b2ae-4b7d-a1a6-d885b1be5fc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:46:21.401944 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 100/208] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:46:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 11:46:21.440368 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d612e8d1-7352-4578-8d9a-f644ee686100 None None] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:21.440738 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d612e8d1-7352-4578-8d9a-f644ee686100 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:21.440960 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d612e8d1-7352-4578-8d9a-f644ee686100 None None] PATH_INFO: `/` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:21.442388 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 101/209] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:46:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 11 11:46:21.448090 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a067474a-17ab-4b5a-a52b-f8319ce8681b None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:21.448440 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a067474a-17ab-4b5a-a52b-f8319ce8681b None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:21.448780 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a067474a-17ab-4b5a-a52b-f8319ce8681b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:21.469361 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-a067474a-17ab-4b5a-a52b-f8319ce8681b None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:46:21.470710 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-a067474a-17ab-4b5a-a52b-f8319ce8681b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:46:21.513070 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 109/210] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:46:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 11 11:46:21.576031 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 102/211] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:46:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 11 11:46:21.670218 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27f517c-b2ae-4b7d-a1a6-d885b1be5fc8 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:21.796511 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27f517c-b2ae-4b7d-a1a6-d885b1be5fc8 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:21.829587 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:21.839730 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20b72f8-7d97-412c-b26a-a5a7a6baee7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:21.840391 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20b72f8-7d97-412c-b26a-a5a7a6baee7e None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:21.840667 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20b72f8-7d97-412c-b26a-a5a7a6baee7e None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:21.840938 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20b72f8-7d97-412c-b26a-a5a7a6baee7e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:21.845767 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a067474a-17ab-4b5a-a52b-f8319ce8681b None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:21.853861 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20b72f8-7d97-412c-b26a-a5a7a6baee7e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 11:46:21.859057 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application [None req-c20b72f8-7d97-412c-b26a-a5a7a6baee7e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 11 11:46:21.859057 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:46:21.859057 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:46:21.859057 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:46:21.859057 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:46:21.859057 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:46:21.859057 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:46:21.859057 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:46:21.859057 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:46:21.859057 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:46:21.859057 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:46:21.859057 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:46:21.859057 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 11:46:21.859057 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 11:46:21.859057 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 11:46:21.859057 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 11:46:21.860862 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 11:46:21.860862 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 11:46:21.860862 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 11:46:21.860862 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 11:46:21.860862 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:46:21.860862 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:46:21.860862 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:46:21.860862 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:46:21.860862 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:46:21.860862 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:46:21.860862 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:46:21.860862 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application with Lock( Oct 11 11:46:21.860862 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:46:21.860862 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:46:21.860862 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:46:21.860862 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:46:21.863199 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:46:21.863199 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:46:21.863199 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:46:21.863199 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:46:21.863199 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 11:46:21.863199 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 11:46:21.863199 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 11:46:21.863199 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 11:46:21.863199 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 11:46:21.863199 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 11:46:21.863199 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 11 11:46:21.863199 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:46:21.863199 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 110/212] 10.222.0.59 () {56 vars in 1099 bytes} [Tue Oct 11 11:46:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 11 11:46:21.867375 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20b72f8-7d97-412c-b26a-a5a7a6baee7e None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:21.980204 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:22.020498 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20b72f8-7d97-412c-b26a-a5a7a6baee7e None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:22.023301 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a067474a-17ab-4b5a-a52b-f8319ce8681b None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:22.063416 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15c3a9a-6076-4f38-9993-7a25355a7f0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:22.064016 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15c3a9a-6076-4f38-9993-7a25355a7f0a None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:22.064299 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15c3a9a-6076-4f38-9993-7a25355a7f0a None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:22.064576 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15c3a9a-6076-4f38-9993-7a25355a7f0a None admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:22.065337 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15c3a9a-6076-4f38-9993-7a25355a7f0a None admin] RBAC: Adding query filter params (name=member) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:46:22.065684 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15c3a9a-6076-4f38-9993-7a25355a7f0a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:22.067373 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15c3a9a-6076-4f38-9993-7a25355a7f0a None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:22.076397 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcb305d-a4c4-4a7c-b1c1-0095c2e4c949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:22.077175 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcb305d-a4c4-4a7c-b1c1-0095c2e4c949 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:22.077502 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcb305d-a4c4-4a7c-b1c1-0095c2e4c949 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:22.077818 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcb305d-a4c4-4a7c-b1c1-0095c2e4c949 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:22.079163 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 111/213] 10.222.0.59 () {56 vars in 1101 bytes} [Tue Oct 11 11:46:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 11:46:22.080360 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15c3a9a-6076-4f38-9993-7a25355a7f0a None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:22.099915 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcb305d-a4c4-4a7c-b1c1-0095c2e4c949 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 11:46:22.108721 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application [None req-bbcb305d-a4c4-4a7c-b1c1-0095c2e4c949 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 11 11:46:22.108721 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:46:22.108721 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:46:22.108721 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:46:22.108721 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:46:22.108721 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:46:22.108721 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:46:22.108721 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:46:22.108721 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:46:22.108721 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:46:22.108721 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:46:22.108721 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:46:22.108721 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 11:46:22.108721 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 11:46:22.108721 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 11:46:22.108721 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 11:46:22.110416 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 11:46:22.110416 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 11:46:22.110416 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 11:46:22.110416 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 11:46:22.110416 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:46:22.110416 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:46:22.110416 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:46:22.110416 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:46:22.110416 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:46:22.110416 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:46:22.110416 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:46:22.110416 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application with Lock( Oct 11 11:46:22.110416 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:46:22.110416 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:46:22.110416 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:46:22.110416 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:46:22.112023 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:46:22.112023 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:46:22.112023 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:46:22.112023 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:46:22.112023 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 11:46:22.112023 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 11:46:22.112023 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 11:46:22.112023 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 11:46:22.112023 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 11:46:22.112023 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 11:46:22.112023 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 11 11:46:22.112023 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:46:22.112023 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 103/214] 10.222.0.59 () {56 vars in 1099 bytes} [Tue Oct 11 11:46:21 2022] GET /identity/v3/roles/reader => generated 84 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:46:22.113396 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcb305d-a4c4-4a7c-b1c1-0095c2e4c949 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:22.165009 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:22.238576 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15c3a9a-6076-4f38-9993-7a25355a7f0a None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:22.239611 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcb305d-a4c4-4a7c-b1c1-0095c2e4c949 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:22.282810 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:22.291710 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5ba764-ace6-47fa-b28e-912afca13dd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:22.292121 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d21069-5197-44cc-be1f-c335709dd490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:22.292570 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5ba764-ace6-47fa-b28e-912afca13dd6 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:22.292919 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d21069-5197-44cc-be1f-c335709dd490 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:22.293103 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5ba764-ace6-47fa-b28e-912afca13dd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:22.293303 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d21069-5197-44cc-be1f-c335709dd490 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:22.293452 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5ba764-ace6-47fa-b28e-912afca13dd6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:22.293693 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d21069-5197-44cc-be1f-c335709dd490 None admin] PATH_INFO: `/v3/users/29cf869e32aa4b5c89a287f3bce9e4c2` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:22.294904 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5ba764-ace6-47fa-b28e-912afca13dd6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:22.296698 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5ba764-ace6-47fa-b28e-912afca13dd6 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:22.299132 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 112/215] 10.222.0.59 () {56 vars in 1108 bytes} [Tue Oct 11 11:46:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:46:22.300534 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5ba764-ace6-47fa-b28e-912afca13dd6 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:22.313591 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d21069-5197-44cc-be1f-c335709dd490 None admin] RBAC: Authorizing `identity:get_user(user_id=29cf869e32aa4b5c89a287f3bce9e4c2)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:22.315912 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d21069-5197-44cc-be1f-c335709dd490 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:22.318996 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 104/216] 10.222.0.59 () {56 vars in 1177 bytes} [Tue Oct 11 11:46:22 2022] GET /identity/v3/users/29cf869e32aa4b5c89a287f3bce9e4c2 => generated 293 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:46:22.320145 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d21069-5197-44cc-be1f-c335709dd490 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:22.448396 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5ba764-ace6-47fa-b28e-912afca13dd6 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:22.448778 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d21069-5197-44cc-be1f-c335709dd490 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:22.468458 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:22.489653 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6dcd65-d288-4dcf-90a8-6b4a170b896e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:22.490437 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6dcd65-d288-4dcf-90a8-6b4a170b896e None admin] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:22.490806 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6dcd65-d288-4dcf-90a8-6b4a170b896e None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:22.491085 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6dcd65-d288-4dcf-90a8-6b4a170b896e None admin] PATH_INFO: `/v3/users` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:22.491742 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6dcd65-d288-4dcf-90a8-6b4a170b896e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:22.491970 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27b2ed3-0adf-4ffe-90a8-4afdfecfa5ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:22.492945 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27b2ed3-0adf-4ffe-90a8-4afdfecfa5ab None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:22.493250 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6dcd65-d288-4dcf-90a8-6b4a170b896e None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:22.493371 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27b2ed3-0adf-4ffe-90a8-4afdfecfa5ab None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:22.493855 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27b2ed3-0adf-4ffe-90a8-4afdfecfa5ab None admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:22.495144 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27b2ed3-0adf-4ffe-90a8-4afdfecfa5ab None admin] RBAC: Adding query filter params (name=reader) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:46:22.495583 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27b2ed3-0adf-4ffe-90a8-4afdfecfa5ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:22.498205 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27b2ed3-0adf-4ffe-90a8-4afdfecfa5ab None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:22.512498 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 113/217] 10.222.0.59 () {56 vars in 1101 bytes} [Tue Oct 11 11:46:22 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:46:22.513708 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27b2ed3-0adf-4ffe-90a8-4afdfecfa5ab None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:22.524600 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 105/218] 10.222.0.59 () {60 vars in 1132 bytes} [Tue Oct 11 11:46:22 2022] POST /identity/v3/users => generated 293 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 11:46:22.525705 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6dcd65-d288-4dcf-90a8-6b4a170b896e None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:22.603333 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:22.703178 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27b2ed3-0adf-4ffe-90a8-4afdfecfa5ab None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:22.705158 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6dcd65-d288-4dcf-90a8-6b4a170b896e None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:22.761911 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2b8ff7-aef7-4bc4-82ea-1f979f34c186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:22.762767 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2b8ff7-aef7-4bc4-82ea-1f979f34c186 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:22.763966 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2b8ff7-aef7-4bc4-82ea-1f979f34c186 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:22.763966 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2b8ff7-aef7-4bc4-82ea-1f979f34c186 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:22.764215 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2b8ff7-aef7-4bc4-82ea-1f979f34c186 None admin] RBAC: Adding query filter params (role.id=de2d2bd26fff4748a79990f97a107164, user.id=29cf869e32aa4b5c89a287f3bce9e4c2, scope.system=all) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:46:22.764357 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2b8ff7-aef7-4bc4-82ea-1f979f34c186 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:22.766423 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2b8ff7-aef7-4bc4-82ea-1f979f34c186 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:22.772339 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 114/219] 10.222.0.59 () {56 vars in 1308 bytes} [Tue Oct 11 11:46:22 2022] GET /identity/v3/role_assignments?user.id=29cf869e32aa4b5c89a287f3bce9e4c2&scope.system=all&role.id=de2d2bd26fff4748a79990f97a107164 => generated 226 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:46:22.772583 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6be983-26fc-497f-9b37-6bee61c3b0cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:22.773196 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6be983-26fc-497f-9b37-6bee61c3b0cc None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:22.773636 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6be983-26fc-497f-9b37-6bee61c3b0cc None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:22.774016 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6be983-26fc-497f-9b37-6bee61c3b0cc None admin] PATH_INFO: `/v3/users/e982a82472774047842211f743f936d9` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:22.776543 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6be983-26fc-497f-9b37-6bee61c3b0cc None admin] RBAC: Authorizing `identity:get_user(user_id=e982a82472774047842211f743f936d9)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:22.778410 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6be983-26fc-497f-9b37-6bee61c3b0cc None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:22.780292 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 106/220] 10.222.0.59 () {56 vars in 1177 bytes} [Tue Oct 11 11:46:22 2022] GET /identity/v3/users/e982a82472774047842211f743f936d9 => generated 297 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:46:22.787229 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2b8ff7-aef7-4bc4-82ea-1f979f34c186 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:22.855573 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:22.947183 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2b8ff7-aef7-4bc4-82ea-1f979f34c186 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:23.013273 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681c89d3-3e17-4adc-a049-7d9876040944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:23.014309 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681c89d3-3e17-4adc-a049-7d9876040944 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:23.014855 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681c89d3-3e17-4adc-a049-7d9876040944 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:23.015332 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681c89d3-3e17-4adc-a049-7d9876040944 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:23.130647 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681c89d3-3e17-4adc-a049-7d9876040944 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:23.133404 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681c89d3-3e17-4adc-a049-7d9876040944 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:23.216780 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:23.317618 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 115/221] 10.222.0.59 () {58 vars in 1282 bytes} [Tue Oct 11 11:46:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 532 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 11 11:46:23.327739 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6be983-26fc-497f-9b37-6bee61c3b0cc None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:23.502821 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6be983-26fc-497f-9b37-6bee61c3b0cc None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:23.536397 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:23.569861 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1606b3f-e447-43b4-8ef2-1c89d77d2a76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:23.571091 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1606b3f-e447-43b4-8ef2-1c89d77d2a76 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:23.571579 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1606b3f-e447-43b4-8ef2-1c89d77d2a76 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:23.572074 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1606b3f-e447-43b4-8ef2-1c89d77d2a76 None admin] PATH_INFO: `/v3/projects/ed233e6a85f94375aece3faac28df210` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:23.574358 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1606b3f-e447-43b4-8ef2-1c89d77d2a76 None admin] RBAC: Authorizing `identity:get_project(project_id=ed233e6a85f94375aece3faac28df210)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:23.577261 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1606b3f-e447-43b4-8ef2-1c89d77d2a76 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:23.580725 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 107/222] 10.222.0.59 () {56 vars in 1186 bytes} [Tue Oct 11 11:46:23 2022] GET /identity/v3/projects/ed233e6a85f94375aece3faac28df210 => generated 302 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:46:23.587531 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681c89d3-3e17-4adc-a049-7d9876040944 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:23.702953 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681c89d3-3e17-4adc-a049-7d9876040944 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:23.742596 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d55136-adf5-4b1f-bb93-dd7a2032d8c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:23.743608 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d55136-adf5-4b1f-bb93-dd7a2032d8c2 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:23.744104 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d55136-adf5-4b1f-bb93-dd7a2032d8c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:23.744595 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d55136-adf5-4b1f-bb93-dd7a2032d8c2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:23.745600 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d55136-adf5-4b1f-bb93-dd7a2032d8c2 None admin] RBAC: Adding query filter params (role.id=acda9dcd3a754265af99c2abc579d2da, scope.project.id=ed233e6a85f94375aece3faac28df210, user.id=e982a82472774047842211f743f936d9) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:46:23.746116 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d55136-adf5-4b1f-bb93-dd7a2032d8c2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:23.747857 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d55136-adf5-4b1f-bb93-dd7a2032d8c2 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:23.755720 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 116/223] 10.222.0.59 () {56 vars in 1374 bytes} [Tue Oct 11 11:46:23 2022] GET /identity/v3/role_assignments?user.id=e982a82472774047842211f743f936d9&scope.project.id=ed233e6a85f94375aece3faac28df210&role.id=acda9dcd3a754265af99c2abc579d2da => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:46:25.073676 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658e9aeb-893c-4bbf-92b5-ab19a13d767e None None] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:25.074296 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658e9aeb-893c-4bbf-92b5-ab19a13d767e None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:25.074296 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658e9aeb-893c-4bbf-92b5-ab19a13d767e None None] PATH_INFO: `/` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:25.076042 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 108/224] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:46:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:46:25.082286 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f1c9a2-7263-4645-89cf-bcf897c5e3b6 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:25.082895 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f1c9a2-7263-4645-89cf-bcf897c5e3b6 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:25.082895 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f1c9a2-7263-4645-89cf-bcf897c5e3b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:25.103950 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-d4f1c9a2-7263-4645-89cf-bcf897c5e3b6 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:46:25.105284 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-d4f1c9a2-7263-4645-89cf-bcf897c5e3b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:46:25.107991 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:25.125378 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c67e0e1-6da2-4fd2-a15e-2cc39690f272 None None] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:25.125666 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c67e0e1-6da2-4fd2-a15e-2cc39690f272 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:25.126099 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c67e0e1-6da2-4fd2-a15e-2cc39690f272 None None] PATH_INFO: `/` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:25.127535 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 109/225] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:46:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 11 11:46:25.133792 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc35c7ac-4c06-4315-bb20-e81521c1e5f4 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:25.134066 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc35c7ac-4c06-4315-bb20-e81521c1e5f4 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:25.134607 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc35c7ac-4c06-4315-bb20-e81521c1e5f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:25.156436 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-fc35c7ac-4c06-4315-bb20-e81521c1e5f4 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:46:25.157782 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-fc35c7ac-4c06-4315-bb20-e81521c1e5f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:46:25.196005 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 117/226] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:46:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 11:46:25.238631 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 110/227] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:46:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 11 11:46:25.265645 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85dc7a2-e4d3-4e90-b7cd-a111fa0db308 None None] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:25.266137 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85dc7a2-e4d3-4e90-b7cd-a111fa0db308 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:25.266645 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85dc7a2-e4d3-4e90-b7cd-a111fa0db308 None None] PATH_INFO: `/` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:25.269222 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 118/228] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:46:25 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:46:25.275409 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc52474-2957-4d6d-9762-e67cf50d3472 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:25.275762 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc52474-2957-4d6d-9762-e67cf50d3472 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:25.276240 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc52474-2957-4d6d-9762-e67cf50d3472 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:25.298271 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-abc52474-2957-4d6d-9762-e67cf50d3472 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:46:25.299652 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-abc52474-2957-4d6d-9762-e67cf50d3472 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:46:25.422951 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 111/229] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:46:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 11:46:25.475102 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85dc7a2-e4d3-4e90-b7cd-a111fa0db308 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:25.505958 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc52474-2957-4d6d-9762-e67cf50d3472 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:25.551434 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:25.590944 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85dc7a2-e4d3-4e90-b7cd-a111fa0db308 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:25.621002 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc52474-2957-4d6d-9762-e67cf50d3472 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:25.628982 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af49835-7679-4db9-bc93-42eafa6a675a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:25.629628 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af49835-7679-4db9-bc93-42eafa6a675a None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:25.629903 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af49835-7679-4db9-bc93-42eafa6a675a None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:25.630177 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af49835-7679-4db9-bc93-42eafa6a675a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:25.638406 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:25.643122 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af49835-7679-4db9-bc93-42eafa6a675a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 11:46:25.648452 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application [None req-0af49835-7679-4db9-bc93-42eafa6a675a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 11 11:46:25.648452 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:46:25.648452 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:46:25.648452 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:46:25.648452 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:46:25.648452 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:46:25.648452 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:46:25.648452 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:46:25.648452 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:46:25.648452 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:46:25.648452 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:46:25.648452 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:46:25.648452 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 11:46:25.648452 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 11:46:25.648452 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 11:46:25.648452 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 11:46:25.650472 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 11:46:25.650472 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 11:46:25.650472 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 11:46:25.650472 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 11:46:25.650472 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:46:25.650472 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:46:25.650472 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:46:25.650472 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:46:25.650472 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:46:25.650472 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:46:25.650472 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:46:25.650472 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application with Lock( Oct 11 11:46:25.650472 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:46:25.650472 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:46:25.650472 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:46:25.650472 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:46:25.652086 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:46:25.652086 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:46:25.652086 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:46:25.652086 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:46:25.652086 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 11:46:25.652086 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 11:46:25.652086 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 11:46:25.652086 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 11:46:25.652086 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 11:46:25.652086 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 11:46:25.652086 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 11 11:46:25.652086 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:46:25.655392 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 119/230] 10.222.0.59 () {56 vars in 1099 bytes} [Tue Oct 11 11:46:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:46:25.661938 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af49835-7679-4db9-bc93-42eafa6a675a None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:25.673164 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1ad2be-bca1-4b5a-8200-c618d1c716e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:25.673863 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1ad2be-bca1-4b5a-8200-c618d1c716e0 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:25.674307 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1ad2be-bca1-4b5a-8200-c618d1c716e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:25.674766 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1ad2be-bca1-4b5a-8200-c618d1c716e0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:25.688075 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1ad2be-bca1-4b5a-8200-c618d1c716e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 11:46:25.693485 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application [None req-9c1ad2be-bca1-4b5a-8200-c618d1c716e0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 11 11:46:25.693485 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:46:25.693485 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:46:25.693485 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:46:25.693485 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:46:25.693485 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:46:25.693485 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:46:25.693485 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:46:25.693485 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:46:25.693485 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:46:25.693485 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:46:25.693485 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:46:25.693485 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 11:46:25.693485 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 11:46:25.693485 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 11:46:25.693485 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 11:46:25.694428 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 11:46:25.694428 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 11:46:25.694428 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 11:46:25.694428 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 11:46:25.694428 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:46:25.694428 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:46:25.694428 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:46:25.694428 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:46:25.694428 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:46:25.694428 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:46:25.694428 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:46:25.694428 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application with Lock( Oct 11 11:46:25.694428 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:46:25.694428 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:46:25.694428 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:46:25.694428 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:46:25.695319 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:46:25.695319 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:46:25.695319 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:46:25.695319 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:46:25.695319 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 11:46:25.695319 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 11:46:25.695319 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 11:46:25.695319 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 11:46:25.695319 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 11:46:25.695319 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 11:46:25.695319 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 11 11:46:25.695319 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:46:25.695319 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 112/231] 10.222.0.59 () {56 vars in 1099 bytes} [Tue Oct 11 11:46:25 2022] GET /identity/v3/roles/reader => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 11 11:46:25.700851 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1ad2be-bca1-4b5a-8200-c618d1c716e0 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:25.797630 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af49835-7679-4db9-bc93-42eafa6a675a None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:25.828917 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:25.833053 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1ad2be-bca1-4b5a-8200-c618d1c716e0 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:25.849180 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d5d6c9-d23f-4d40-bc48-f5fdcc517c5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:25.849918 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d5d6c9-d23f-4d40-bc48-f5fdcc517c5a None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:25.850205 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d5d6c9-d23f-4d40-bc48-f5fdcc517c5a None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:25.850504 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d5d6c9-d23f-4d40-bc48-f5fdcc517c5a None admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:25.851499 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d5d6c9-d23f-4d40-bc48-f5fdcc517c5a None admin] RBAC: Adding query filter params (name=member) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:46:25.851871 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d5d6c9-d23f-4d40-bc48-f5fdcc517c5a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:25.853779 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d5d6c9-d23f-4d40-bc48-f5fdcc517c5a None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:25.869075 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 120/232] 10.222.0.59 () {56 vars in 1101 bytes} [Tue Oct 11 11:46:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 11:46:25.870618 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d5d6c9-d23f-4d40-bc48-f5fdcc517c5a None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:25.874228 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdd8b0a-1831-4319-bd20-77191492e16e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:25.874846 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdd8b0a-1831-4319-bd20-77191492e16e None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:25.875156 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdd8b0a-1831-4319-bd20-77191492e16e None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:25.875436 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdd8b0a-1831-4319-bd20-77191492e16e None admin] PATH_INFO: `/v3/roles` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:25.876117 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdd8b0a-1831-4319-bd20-77191492e16e None admin] RBAC: Adding query filter params (name=reader) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:46:25.876423 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdd8b0a-1831-4319-bd20-77191492e16e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:25.877953 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdd8b0a-1831-4319-bd20-77191492e16e None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:25.886945 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 113/233] 10.222.0.59 () {56 vars in 1101 bytes} [Tue Oct 11 11:46:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 11:46:25.888704 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdd8b0a-1831-4319-bd20-77191492e16e None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:25.963772 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:26.008503 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d5d6c9-d23f-4d40-bc48-f5fdcc517c5a None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:26.017708 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdd8b0a-1831-4319-bd20-77191492e16e None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:26.045444 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6797eca1-5f81-4167-a284-e1a0d7fd52f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:26.046086 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6797eca1-5f81-4167-a284-e1a0d7fd52f9 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:26.046362 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6797eca1-5f81-4167-a284-e1a0d7fd52f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:26.046687 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6797eca1-5f81-4167-a284-e1a0d7fd52f9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:26.047923 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6797eca1-5f81-4167-a284-e1a0d7fd52f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:26.049382 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6797eca1-5f81-4167-a284-e1a0d7fd52f9 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:26.051106 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 121/234] 10.222.0.59 () {56 vars in 1108 bytes} [Tue Oct 11 11:46:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:46:26.051946 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6797eca1-5f81-4167-a284-e1a0d7fd52f9 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:26.054015 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0ad2ec-abbe-41cb-81c8-e17c42855e3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:26.054638 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0ad2ec-abbe-41cb-81c8-e17c42855e3e None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:26.054925 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0ad2ec-abbe-41cb-81c8-e17c42855e3e None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:26.055202 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0ad2ec-abbe-41cb-81c8-e17c42855e3e None admin] PATH_INFO: `/v3/users/29cf869e32aa4b5c89a287f3bce9e4c2` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:26.056708 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0ad2ec-abbe-41cb-81c8-e17c42855e3e None admin] RBAC: Authorizing `identity:get_user(user_id=29cf869e32aa4b5c89a287f3bce9e4c2)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:26.058275 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0ad2ec-abbe-41cb-81c8-e17c42855e3e None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:26.059927 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 114/235] 10.222.0.59 () {56 vars in 1177 bytes} [Tue Oct 11 11:46:25 2022] GET /identity/v3/users/29cf869e32aa4b5c89a287f3bce9e4c2 => generated 293 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:46:26.060687 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0ad2ec-abbe-41cb-81c8-e17c42855e3e None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:26.100433 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:26.182741 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6797eca1-5f81-4167-a284-e1a0d7fd52f9 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:26.205859 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0ad2ec-abbe-41cb-81c8-e17c42855e3e None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:26.214321 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:26.224059 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f848fb3e-45f3-4b45-9b8d-ae038a126fc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:26.224734 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f848fb3e-45f3-4b45-9b8d-ae038a126fc5 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:26.225012 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f848fb3e-45f3-4b45-9b8d-ae038a126fc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:26.225286 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f848fb3e-45f3-4b45-9b8d-ae038a126fc5 None admin] PATH_INFO: `/v3/users/0a837f6cfa85408a8a466d293da5b5b2` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:26.244427 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f848fb3e-45f3-4b45-9b8d-ae038a126fc5 None admin] RBAC: Authorizing `identity:get_user(user_id=0a837f6cfa85408a8a466d293da5b5b2)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:26.245893 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6284fd49-0ff3-4294-8b7f-f57b196f2503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:26.246537 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6284fd49-0ff3-4294-8b7f-f57b196f2503 None admin] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:26.246839 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6284fd49-0ff3-4294-8b7f-f57b196f2503 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:26.247124 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f848fb3e-45f3-4b45-9b8d-ae038a126fc5 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:26.247281 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6284fd49-0ff3-4294-8b7f-f57b196f2503 None admin] PATH_INFO: `/v3/groups` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:26.247780 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6284fd49-0ff3-4294-8b7f-f57b196f2503 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:26.249242 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6284fd49-0ff3-4294-8b7f-f57b196f2503 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:26.255739 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 122/236] 10.222.0.59 () {56 vars in 1177 bytes} [Tue Oct 11 11:46:26 2022] GET /identity/v3/users/0a837f6cfa85408a8a466d293da5b5b2 => generated 293 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:46:26.255739 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f848fb3e-45f3-4b45-9b8d-ae038a126fc5 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:26.277188 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 115/237] 10.222.0.59 () {60 vars in 1134 bytes} [Tue Oct 11 11:46:26 2022] POST /identity/v3/groups => generated 229 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 11:46:26.278808 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6284fd49-0ff3-4294-8b7f-f57b196f2503 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:26.407391 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:26.422825 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6284fd49-0ff3-4294-8b7f-f57b196f2503 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:26.437469 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f848fb3e-45f3-4b45-9b8d-ae038a126fc5 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:26.464502 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739f9789-b141-4069-a620-ab2c09510365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:26.465165 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739f9789-b141-4069-a620-ab2c09510365 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:26.465502 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739f9789-b141-4069-a620-ab2c09510365 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:26.465802 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739f9789-b141-4069-a620-ab2c09510365 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:26.466607 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739f9789-b141-4069-a620-ab2c09510365 None admin] RBAC: Adding query filter params (role.id=acda9dcd3a754265af99c2abc579d2da, user.id=0a837f6cfa85408a8a466d293da5b5b2, scope.system=all) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:46:26.466925 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739f9789-b141-4069-a620-ab2c09510365 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:26.468528 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739f9789-b141-4069-a620-ab2c09510365 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:26.475258 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 116/238] 10.222.0.59 () {56 vars in 1308 bytes} [Tue Oct 11 11:46:26 2022] GET /identity/v3/role_assignments?user.id=0a837f6cfa85408a8a466d293da5b5b2&scope.system=all&role.id=acda9dcd3a754265af99c2abc579d2da => generated 226 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:46:26.504225 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314e1ad5-0c23-4bd1-8871-3479ba630aec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:26.504989 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314e1ad5-0c23-4bd1-8871-3479ba630aec None admin] REQUEST_METHOD: `PUT` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:26.505384 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314e1ad5-0c23-4bd1-8871-3479ba630aec None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:26.505753 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314e1ad5-0c23-4bd1-8871-3479ba630aec None admin] PATH_INFO: `/v3/system/users/29cf869e32aa4b5c89a287f3bce9e4c2/roles/de2d2bd26fff4748a79990f97a107164` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:26.507736 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314e1ad5-0c23-4bd1-8871-3479ba630aec None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=29cf869e32aa4b5c89a287f3bce9e4c2, role_id=de2d2bd26fff4748a79990f97a107164)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:26.509274 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314e1ad5-0c23-4bd1-8871-3479ba630aec None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:26.522069 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 123/239] 10.222.0.59 () {58 vars in 1334 bytes} [Tue Oct 11 11:46:26 2022] PUT /identity/v3/system/users/29cf869e32aa4b5c89a287f3bce9e4c2/roles/de2d2bd26fff4748a79990f97a107164 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 11:46:28.589962 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a8aeeb-7d28-4ffa-af82-f79a29e59133 None None] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:28.590674 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a8aeeb-7d28-4ffa-af82-f79a29e59133 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:28.590674 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a8aeeb-7d28-4ffa-af82-f79a29e59133 None None] PATH_INFO: `/` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:28.594626 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 117/240] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:46:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:46:28.598008 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6ebfcf-0d1c-48b8-81d0-ab210d4a85ce None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:28.598850 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6ebfcf-0d1c-48b8-81d0-ab210d4a85ce None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:28.598850 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6ebfcf-0d1c-48b8-81d0-ab210d4a85ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:28.620135 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-6a6ebfcf-0d1c-48b8-81d0-ab210d4a85ce None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:46:28.621447 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-6a6ebfcf-0d1c-48b8-81d0-ab210d4a85ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:46:28.681186 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:28.685133 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b2b52a-578e-43cf-879b-43db30a5f8f3 None None] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:28.685431 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b2b52a-578e-43cf-879b-43db30a5f8f3 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:28.685687 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b2b52a-578e-43cf-879b-43db30a5f8f3 None None] PATH_INFO: `/` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:28.687366 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 118/241] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:46:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 11 11:46:28.693419 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f0059d-4904-4978-8790-d4ac7c059ec8 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:28.693702 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f0059d-4904-4978-8790-d4ac7c059ec8 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:28.694042 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f0059d-4904-4978-8790-d4ac7c059ec8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:28.713656 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 124/242] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:46:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 11 11:46:28.716680 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-49f0059d-4904-4978-8790-d4ac7c059ec8 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:46:28.717918 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-49f0059d-4904-4978-8790-d4ac7c059ec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:46:28.749333 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7907c3ea-160e-4eb6-9563-18e50b60448c None None] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:28.749617 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7907c3ea-160e-4eb6-9563-18e50b60448c None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:28.749915 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7907c3ea-160e-4eb6-9563-18e50b60448c None None] PATH_INFO: `/` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:28.751363 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 125/243] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:46:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 11 11:46:28.757192 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e2f926-df03-4b34-9252-9a4cf070c5a9 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:28.757477 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e2f926-df03-4b34-9252-9a4cf070c5a9 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:28.757753 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e2f926-df03-4b34-9252-9a4cf070c5a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:28.777861 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-c9e2f926-df03-4b34-9252-9a4cf070c5a9 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:46:28.779153 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-c9e2f926-df03-4b34-9252-9a4cf070c5a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:46:28.810284 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 119/244] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:46:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 11 11:46:28.856893 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 126/245] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:46:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 11 11:46:28.989979 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f0059d-4904-4978-8790-d4ac7c059ec8 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:29.074959 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e2f926-df03-4b34-9252-9a4cf070c5a9 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:29.134331 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:29.158488 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f0059d-4904-4978-8790-d4ac7c059ec8 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:29.188018 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e2f926-df03-4b34-9252-9a4cf070c5a9 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:29.200341 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:29.209926 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2a7369-0f0b-4b86-b22a-ee40ef0678bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:29.210624 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2a7369-0f0b-4b86-b22a-ee40ef0678bd None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:29.210907 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2a7369-0f0b-4b86-b22a-ee40ef0678bd None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:29.211179 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2a7369-0f0b-4b86-b22a-ee40ef0678bd None admin] PATH_INFO: `/v3/domains/default` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:29.213094 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2a7369-0f0b-4b86-b22a-ee40ef0678bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:29.214685 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2a7369-0f0b-4b86-b22a-ee40ef0678bd None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:29.216437 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 120/246] 10.222.0.59 () {56 vars in 1108 bytes} [Tue Oct 11 11:46:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 11:46:29.217300 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2a7369-0f0b-4b86-b22a-ee40ef0678bd None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:29.233431 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1334928a-e946-4dc7-b1ba-b5b420eb700b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:29.234144 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1334928a-e946-4dc7-b1ba-b5b420eb700b None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:29.234422 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1334928a-e946-4dc7-b1ba-b5b420eb700b None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:29.234734 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1334928a-e946-4dc7-b1ba-b5b420eb700b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:29.246290 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1334928a-e946-4dc7-b1ba-b5b420eb700b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 11:46:29.251284 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application [None req-1334928a-e946-4dc7-b1ba-b5b420eb700b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 11 11:46:29.251284 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:46:29.251284 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:46:29.251284 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:46:29.251284 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:46:29.251284 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:46:29.251284 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:46:29.251284 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:46:29.251284 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:46:29.251284 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:46:29.251284 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:46:29.251284 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:46:29.251284 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 11:46:29.251284 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 11:46:29.251284 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 11:46:29.251284 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 11:46:29.253064 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 11:46:29.253064 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 11:46:29.253064 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 11:46:29.253064 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 11:46:29.253064 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:46:29.253064 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:46:29.253064 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:46:29.253064 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:46:29.253064 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:46:29.253064 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:46:29.253064 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:46:29.253064 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application with Lock( Oct 11 11:46:29.253064 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:46:29.253064 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:46:29.253064 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:46:29.253064 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:46:29.254611 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:46:29.254611 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:46:29.254611 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:46:29.254611 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:46:29.254611 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 11:46:29.254611 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 11:46:29.254611 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 11:46:29.254611 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 11:46:29.254611 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 11:46:29.254611 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 11:46:29.254611 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 11 11:46:29.254611 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:46:29.254611 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 127/247] 10.222.0.59 () {56 vars in 1099 bytes} [Tue Oct 11 11:46:29 2022] GET /identity/v3/roles/reader => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 11 11:46:29.257900 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1334928a-e946-4dc7-b1ba-b5b420eb700b None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:29.349810 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2a7369-0f0b-4b86-b22a-ee40ef0678bd None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:29.388167 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150c362e-6600-49fa-a6e7-f0cc964ab4fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:29.388697 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150c362e-6600-49fa-a6e7-f0cc964ab4fa None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:29.388980 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150c362e-6600-49fa-a6e7-f0cc964ab4fa None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:29.389250 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150c362e-6600-49fa-a6e7-f0cc964ab4fa None admin] PATH_INFO: `/v3/roles/member` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:29.404774 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1334928a-e946-4dc7-b1ba-b5b420eb700b None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:29.405671 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150c362e-6600-49fa-a6e7-f0cc964ab4fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 11:46:29.415813 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application [None req-150c362e-6600-49fa-a6e7-f0cc964ab4fa None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 11 11:46:29.415813 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:46:29.415813 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:46:29.415813 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:46:29.415813 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:46:29.415813 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:46:29.415813 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:46:29.415813 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:46:29.415813 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:46:29.415813 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:46:29.415813 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:46:29.415813 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:46:29.415813 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 11:46:29.415813 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 11:46:29.415813 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 11:46:29.415813 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 11:46:29.417584 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 11:46:29.417584 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 11:46:29.417584 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 11:46:29.417584 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 11:46:29.417584 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:46:29.417584 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:46:29.417584 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:46:29.417584 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:46:29.417584 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:46:29.417584 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:46:29.417584 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:46:29.417584 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application with Lock( Oct 11 11:46:29.417584 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:46:29.417584 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:46:29.417584 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:46:29.417584 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:46:29.419188 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:46:29.419188 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:46:29.419188 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:46:29.419188 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:46:29.419188 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 11:46:29.419188 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 11:46:29.419188 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 11:46:29.419188 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 11:46:29.419188 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 11:46:29.419188 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 11:46:29.419188 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 11 11:46:29.419188 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:46:29.419188 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 121/248] 10.222.0.59 () {56 vars in 1099 bytes} [Tue Oct 11 11:46:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:46:29.419188 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150c362e-6600-49fa-a6e7-f0cc964ab4fa None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:29.438155 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:29.448945 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09ccdff-cf7a-4ec7-badb-45240c505bf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:29.449582 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09ccdff-cf7a-4ec7-badb-45240c505bf6 None admin] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:29.449862 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09ccdff-cf7a-4ec7-badb-45240c505bf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:29.450136 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09ccdff-cf7a-4ec7-badb-45240c505bf6 None admin] PATH_INFO: `/v3/groups` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:29.450805 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09ccdff-cf7a-4ec7-badb-45240c505bf6 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:29.452291 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09ccdff-cf7a-4ec7-badb-45240c505bf6 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:29.480525 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 128/249] 10.222.0.59 () {60 vars in 1134 bytes} [Tue Oct 11 11:46:29 2022] POST /identity/v3/groups => generated 226 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 11:46:29.481988 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09ccdff-cf7a-4ec7-badb-45240c505bf6 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:29.543923 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:29.564255 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150c362e-6600-49fa-a6e7-f0cc964ab4fa None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:29.605407 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49aac660-60d3-4dea-82c6-42eedd76fb5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:29.606455 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aac660-60d3-4dea-82c6-42eedd76fb5c None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:29.606455 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aac660-60d3-4dea-82c6-42eedd76fb5c None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:29.607169 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aac660-60d3-4dea-82c6-42eedd76fb5c None admin] PATH_INFO: `/v3/roles` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:29.607331 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49aac660-60d3-4dea-82c6-42eedd76fb5c None admin] RBAC: Adding query filter params (name=reader) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:46:29.607705 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49aac660-60d3-4dea-82c6-42eedd76fb5c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:29.609017 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49aac660-60d3-4dea-82c6-42eedd76fb5c None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:29.619136 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 122/250] 10.222.0.59 () {56 vars in 1101 bytes} [Tue Oct 11 11:46:29 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:46:29.624918 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49aac660-60d3-4dea-82c6-42eedd76fb5c None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:29.686418 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09ccdff-cf7a-4ec7-badb-45240c505bf6 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:29.732048 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:29.746983 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9140fbd1-fce6-41eb-b7fd-19f2c1cfc734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:29.747629 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9140fbd1-fce6-41eb-b7fd-19f2c1cfc734 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:29.747910 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9140fbd1-fce6-41eb-b7fd-19f2c1cfc734 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:29.748187 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9140fbd1-fce6-41eb-b7fd-19f2c1cfc734 None admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:29.748904 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9140fbd1-fce6-41eb-b7fd-19f2c1cfc734 None admin] RBAC: Adding query filter params (name=member) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:46:29.749213 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9140fbd1-fce6-41eb-b7fd-19f2c1cfc734 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:29.750747 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9140fbd1-fce6-41eb-b7fd-19f2c1cfc734 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:29.771160 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 129/251] 10.222.0.59 () {56 vars in 1101 bytes} [Tue Oct 11 11:46:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:46:29.778185 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9140fbd1-fce6-41eb-b7fd-19f2c1cfc734 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:29.778785 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49aac660-60d3-4dea-82c6-42eedd76fb5c None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:29.851034 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff02909-97c6-4479-8750-6062504b6429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:29.852078 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff02909-97c6-4479-8750-6062504b6429 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:29.852547 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff02909-97c6-4479-8750-6062504b6429 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:29.853234 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff02909-97c6-4479-8750-6062504b6429 None admin] PATH_INFO: `/v3/users/0a837f6cfa85408a8a466d293da5b5b2` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:29.855662 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff02909-97c6-4479-8750-6062504b6429 None admin] RBAC: Authorizing `identity:get_user(user_id=0a837f6cfa85408a8a466d293da5b5b2)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:29.858344 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff02909-97c6-4479-8750-6062504b6429 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:29.869541 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 123/252] 10.222.0.59 () {56 vars in 1177 bytes} [Tue Oct 11 11:46:29 2022] GET /identity/v3/users/0a837f6cfa85408a8a466d293da5b5b2 => generated 293 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 11:46:29.873723 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff02909-97c6-4479-8750-6062504b6429 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:29.981447 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:29.994848 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9140fbd1-fce6-41eb-b7fd-19f2c1cfc734 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:30.057575 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919f70e5-e559-4be9-809f-9981c6333244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:30.058681 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919f70e5-e559-4be9-809f-9981c6333244 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:30.058928 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff02909-97c6-4479-8750-6062504b6429 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:30.059455 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919f70e5-e559-4be9-809f-9981c6333244 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:30.059940 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919f70e5-e559-4be9-809f-9981c6333244 None admin] PATH_INFO: `/v3/users/29cf869e32aa4b5c89a287f3bce9e4c2` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:30.062315 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919f70e5-e559-4be9-809f-9981c6333244 None admin] RBAC: Authorizing `identity:get_user(user_id=29cf869e32aa4b5c89a287f3bce9e4c2)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:30.063937 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919f70e5-e559-4be9-809f-9981c6333244 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:30.065963 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 130/253] 10.222.0.59 () {56 vars in 1177 bytes} [Tue Oct 11 11:46:29 2022] GET /identity/v3/users/29cf869e32aa4b5c89a287f3bce9e4c2 => generated 293 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 11:46:30.072236 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919f70e5-e559-4be9-809f-9981c6333244 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:30.127386 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396ed942-6bb7-41ec-844d-c5b7d14d5420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:30.129350 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396ed942-6bb7-41ec-844d-c5b7d14d5420 None admin] REQUEST_METHOD: `PUT` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:30.130208 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396ed942-6bb7-41ec-844d-c5b7d14d5420 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:30.131007 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396ed942-6bb7-41ec-844d-c5b7d14d5420 None admin] PATH_INFO: `/v3/system/users/0a837f6cfa85408a8a466d293da5b5b2/roles/acda9dcd3a754265af99c2abc579d2da` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:30.134222 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396ed942-6bb7-41ec-844d-c5b7d14d5420 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=0a837f6cfa85408a8a466d293da5b5b2, role_id=acda9dcd3a754265af99c2abc579d2da)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:30.137293 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396ed942-6bb7-41ec-844d-c5b7d14d5420 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:30.158447 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 124/254] 10.222.0.59 () {58 vars in 1334 bytes} [Tue Oct 11 11:46:29 2022] PUT /identity/v3/system/users/0a837f6cfa85408a8a466d293da5b5b2/roles/acda9dcd3a754265af99c2abc579d2da => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 11:46:30.217315 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919f70e5-e559-4be9-809f-9981c6333244 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:30.265317 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cece3081-c9f1-4d2c-be25-7c8d935c1762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:30.265839 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cece3081-c9f1-4d2c-be25-7c8d935c1762 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:30.266116 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cece3081-c9f1-4d2c-be25-7c8d935c1762 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:30.266437 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cece3081-c9f1-4d2c-be25-7c8d935c1762 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:30.267290 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cece3081-c9f1-4d2c-be25-7c8d935c1762 None admin] RBAC: Adding query filter params (role.id=de2d2bd26fff4748a79990f97a107164, user.id=29cf869e32aa4b5c89a287f3bce9e4c2, scope.system=all) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:46:30.267606 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cece3081-c9f1-4d2c-be25-7c8d935c1762 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:30.269187 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cece3081-c9f1-4d2c-be25-7c8d935c1762 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:30.285039 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 131/255] 10.222.0.59 () {56 vars in 1308 bytes} [Tue Oct 11 11:46:30 2022] GET /identity/v3/role_assignments?user.id=29cf869e32aa4b5c89a287f3bce9e4c2&scope.system=all&role.id=de2d2bd26fff4748a79990f97a107164 => generated 510 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:46:32.965473 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eec813b-5b15-4c84-8548-7d08e2708094 None None] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:32.966865 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eec813b-5b15-4c84-8548-7d08e2708094 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:32.967561 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eec813b-5b15-4c84-8548-7d08e2708094 None None] PATH_INFO: `/` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:32.970237 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 125/256] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:46:32 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:46:32.977647 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6d9a08-fd50-4a23-b4f2-d503bf3694ac None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:32.977647 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6d9a08-fd50-4a23-b4f2-d503bf3694ac None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:32.977647 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6d9a08-fd50-4a23-b4f2-d503bf3694ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:32.998627 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5af50b-5f22-4d97-9739-562a0835f9cc None None] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:32.999134 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5af50b-5f22-4d97-9739-562a0835f9cc None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:32.999627 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5af50b-5f22-4d97-9739-562a0835f9cc None None] PATH_INFO: `/` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:33.006504 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 126/257] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:46:32 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:46:33.006925 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-7d6d9a08-fd50-4a23-b4f2-d503bf3694ac None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:46:33.008241 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-7d6d9a08-fd50-4a23-b4f2-d503bf3694ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:46:33.009094 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ef6a3e-b65a-4ba4-9b15-f9a2b958433a None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:33.009593 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ef6a3e-b65a-4ba4-9b15-f9a2b958433a None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:33.010128 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ef6a3e-b65a-4ba4-9b15-f9a2b958433a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:33.047159 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:33.050703 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:33.068502 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-e6ef6a3e-b65a-4ba4-9b15-f9a2b958433a None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:46:33.070679 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-e6ef6a3e-b65a-4ba4-9b15-f9a2b958433a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:46:33.169429 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 132/258] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:46:32 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 11 11:46:33.258856 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 127/259] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:46:33 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Oct 11 11:46:33.608932 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6d9a08-fd50-4a23-b4f2-d503bf3694ac None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:33.619157 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6d9a08-fd50-4a23-b4f2-d503bf3694ac None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:33.631337 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7f4b34-6389-4f20-b915-f2a8fac1a104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:33.632007 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7f4b34-6389-4f20-b915-f2a8fac1a104 admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:33.632290 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7f4b34-6389-4f20-b915-f2a8fac1a104 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:33.632565 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7f4b34-6389-4f20-b915-f2a8fac1a104 admin admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:33.649298 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7f4b34-6389-4f20-b915-f2a8fac1a104 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 11:46:33.654375 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application [None req-bc7f4b34-6389-4f20-b915-f2a8fac1a104 admin admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 11 11:46:33.654375 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:46:33.654375 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:46:33.654375 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:46:33.654375 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:46:33.654375 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:46:33.654375 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:46:33.654375 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:46:33.654375 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:46:33.654375 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:46:33.654375 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:46:33.654375 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:46:33.654375 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 11:46:33.654375 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 11:46:33.654375 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 11:46:33.654375 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 11:46:33.656134 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 11:46:33.656134 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 11:46:33.656134 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 11:46:33.656134 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 11:46:33.656134 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:46:33.656134 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:46:33.656134 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:46:33.656134 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:46:33.656134 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:46:33.656134 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:46:33.656134 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:46:33.656134 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application with Lock( Oct 11 11:46:33.656134 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:46:33.656134 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:46:33.656134 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:46:33.656134 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:46:33.657646 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:46:33.657646 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:46:33.657646 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:46:33.657646 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:46:33.657646 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 11:46:33.657646 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 11:46:33.657646 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 11:46:33.657646 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 11:46:33.657646 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 11:46:33.657646 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 11:46:33.657646 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 11 11:46:33.657646 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:46:33.657646 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 133/260] 10.222.0.59 () {56 vars in 1135 bytes} [Tue Oct 11 11:46:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 47 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:46:33.662420 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ef6a3e-b65a-4ba4-9b15-f9a2b958433a None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:33.677046 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ef6a3e-b65a-4ba4-9b15-f9a2b958433a None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:33.685921 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f819c740-bf04-43e4-8723-2be47866c20c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:33.686667 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f819c740-bf04-43e4-8723-2be47866c20c admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:33.686959 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f819c740-bf04-43e4-8723-2be47866c20c admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:33.687203 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f819c740-bf04-43e4-8723-2be47866c20c admin admin] PATH_INFO: `/v3/roles` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:33.687933 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f819c740-bf04-43e4-8723-2be47866c20c admin admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:46:33.688255 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f819c740-bf04-43e4-8723-2be47866c20c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:33.690347 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: WARNING py.warnings [None req-f819c740-bf04-43e4-8723-2be47866c20c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 11 11:46:33.690347 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: warnings.warn(msg) Oct 11 11:46:33.690347 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: Oct 11 11:46:33.691404 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f819c740-bf04-43e4-8723-2be47866c20c admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:33.708533 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 128/261] 10.222.0.59 () {56 vars in 1132 bytes} [Tue Oct 11 11:46:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:46:33.714289 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7f4b34-6389-4f20-b915-f2a8fac1a104 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:33.728471 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7f4b34-6389-4f20-b915-f2a8fac1a104 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:33.736454 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb88cfa-5086-492e-9d72-2f227515af2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:33.737178 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb88cfa-5086-492e-9d72-2f227515af2b admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:33.737438 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb88cfa-5086-492e-9d72-2f227515af2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:33.737797 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb88cfa-5086-492e-9d72-2f227515af2b admin admin] PATH_INFO: `/v3/groups/5e917abb7b794d318682da2487edff33` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:33.751417 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb88cfa-5086-492e-9d72-2f227515af2b admin admin] RBAC: Authorizing `identity:get_group(group_id=5e917abb7b794d318682da2487edff33)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:33.753397 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: WARNING py.warnings [None req-3bb88cfa-5086-492e-9d72-2f227515af2b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 11 11:46:33.753397 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: warnings.warn(msg) Oct 11 11:46:33.753397 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: Oct 11 11:46:33.754687 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb88cfa-5086-492e-9d72-2f227515af2b admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:33.756562 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 134/262] 10.222.0.59 () {56 vars in 1201 bytes} [Tue Oct 11 11:46:33 2022] GET /identity/v3/groups/5e917abb7b794d318682da2487edff33 => generated 226 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 11:46:33.762258 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f819c740-bf04-43e4-8723-2be47866c20c admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:33.776791 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f819c740-bf04-43e4-8723-2be47866c20c admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:33.786398 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0f7d60-f3cb-49dc-8b39-edd2c4e693bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:33.787099 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0f7d60-f3cb-49dc-8b39-edd2c4e693bc admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:33.787389 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0f7d60-f3cb-49dc-8b39-edd2c4e693bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:33.787674 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0f7d60-f3cb-49dc-8b39-edd2c4e693bc admin admin] PATH_INFO: `/v3/projects/1a4b3d97a66e4af4be0a066285f102de` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:33.791394 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0f7d60-f3cb-49dc-8b39-edd2c4e693bc admin admin] RBAC: Authorizing `identity:get_project(project_id=1a4b3d97a66e4af4be0a066285f102de)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:33.793684 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0f7d60-f3cb-49dc-8b39-edd2c4e693bc admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:33.795677 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 129/263] 10.222.0.59 () {56 vars in 1207 bytes} [Tue Oct 11 11:46:33 2022] GET /identity/v3/projects/1a4b3d97a66e4af4be0a066285f102de => generated 298 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:46:33.801971 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb88cfa-5086-492e-9d72-2f227515af2b admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:33.817175 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb88cfa-5086-492e-9d72-2f227515af2b admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:33.825433 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca0984f-372d-4565-abc2-86dc2ac3cb1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:33.826153 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca0984f-372d-4565-abc2-86dc2ac3cb1d admin admin] REQUEST_METHOD: `PUT` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:33.826415 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca0984f-372d-4565-abc2-86dc2ac3cb1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:33.826724 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca0984f-372d-4565-abc2-86dc2ac3cb1d admin admin] PATH_INFO: `/v3/projects/1a4b3d97a66e4af4be0a066285f102de/groups/5e917abb7b794d318682da2487edff33/roles/3a1a5476791547419a0b885c2022ef68` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:33.832949 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca0984f-372d-4565-abc2-86dc2ac3cb1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a4b3d97a66e4af4be0a066285f102de, group_id=5e917abb7b794d318682da2487edff33, role_id=3a1a5476791547419a0b885c2022ef68)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:33.834080 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: WARNING py.warnings [None req-bca0984f-372d-4565-abc2-86dc2ac3cb1d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 11 11:46:33.834080 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: warnings.warn(msg) Oct 11 11:46:33.834080 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: Oct 11 11:46:33.836185 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca0984f-372d-4565-abc2-86dc2ac3cb1d admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:33.851463 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0f7d60-f3cb-49dc-8b39-edd2c4e693bc admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:33.863750 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 135/264] 10.222.0.59 () {58 vars in 1463 bytes} [Tue Oct 11 11:46:33 2022] PUT /identity/v3/projects/1a4b3d97a66e4af4be0a066285f102de/groups/5e917abb7b794d318682da2487edff33/roles/3a1a5476791547419a0b885c2022ef68 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 11:46:33.871625 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0f7d60-f3cb-49dc-8b39-edd2c4e693bc admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:33.975128 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b532574-c518-41f1-8f83-fb63d61b4567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:33.977505 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b532574-c518-41f1-8f83-fb63d61b4567 admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:33.978285 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b532574-c518-41f1-8f83-fb63d61b4567 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:33.979169 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b532574-c518-41f1-8f83-fb63d61b4567 admin admin] PATH_INFO: `/v3/roles/member` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:33.999852 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b532574-c518-41f1-8f83-fb63d61b4567 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 11:46:34.011768 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application [None req-3b532574-c518-41f1-8f83-fb63d61b4567 admin admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 11 11:46:34.011768 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:46:34.011768 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:46:34.011768 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:46:34.011768 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:46:34.011768 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:46:34.011768 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:46:34.011768 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:46:34.011768 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:46:34.011768 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:46:34.011768 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:46:34.011768 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:46:34.011768 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 11:46:34.011768 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 11:46:34.011768 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 11:46:34.011768 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 11:46:34.013501 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 11:46:34.013501 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 11:46:34.013501 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 11:46:34.013501 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 11:46:34.013501 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:46:34.013501 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:46:34.013501 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:46:34.013501 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:46:34.013501 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:46:34.013501 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:46:34.013501 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:46:34.013501 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application with Lock( Oct 11 11:46:34.013501 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:46:34.013501 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:46:34.013501 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:46:34.013501 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:46:34.015130 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:46:34.015130 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:46:34.015130 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:46:34.015130 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:46:34.015130 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 11:46:34.015130 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 11:46:34.015130 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 11:46:34.015130 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 11:46:34.015130 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 11:46:34.015130 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 11:46:34.015130 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 11 11:46:34.015130 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:46:34.020222 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 130/265] 10.222.0.59 () {56 vars in 1120 bytes} [Tue Oct 11 11:46:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 11 11:46:34.026550 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca0984f-372d-4565-abc2-86dc2ac3cb1d admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:34.045375 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca0984f-372d-4565-abc2-86dc2ac3cb1d admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:34.060905 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303d3c67-4f78-472c-8c4e-6ac5318418b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:34.061568 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303d3c67-4f78-472c-8c4e-6ac5318418b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:34.061871 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303d3c67-4f78-472c-8c4e-6ac5318418b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:34.062124 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303d3c67-4f78-472c-8c4e-6ac5318418b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:34.062859 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303d3c67-4f78-472c-8c4e-6ac5318418b4 admin admin] RBAC: Adding query filter params (name=member) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:46:34.063167 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303d3c67-4f78-472c-8c4e-6ac5318418b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:34.064266 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: WARNING py.warnings [None req-303d3c67-4f78-472c-8c4e-6ac5318418b4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 11 11:46:34.064266 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: warnings.warn(msg) Oct 11 11:46:34.064266 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: Oct 11 11:46:34.065249 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303d3c67-4f78-472c-8c4e-6ac5318418b4 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:34.080148 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 136/266] 10.222.0.59 () {56 vars in 1122 bytes} [Tue Oct 11 11:46:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:46:34.087148 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b532574-c518-41f1-8f83-fb63d61b4567 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:34.108298 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b532574-c518-41f1-8f83-fb63d61b4567 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:34.116953 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f234c23-e561-408d-9567-ed7b18460f31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:34.118588 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f234c23-e561-408d-9567-ed7b18460f31 admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:34.119492 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f234c23-e561-408d-9567-ed7b18460f31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:34.120189 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f234c23-e561-408d-9567-ed7b18460f31 admin admin] PATH_INFO: `/v3/groups/5e917abb7b794d318682da2487edff33` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:34.122731 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f234c23-e561-408d-9567-ed7b18460f31 admin admin] RBAC: Authorizing `identity:get_group(group_id=5e917abb7b794d318682da2487edff33)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:34.125061 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: WARNING py.warnings [None req-4f234c23-e561-408d-9567-ed7b18460f31 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 11 11:46:34.125061 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: warnings.warn(msg) Oct 11 11:46:34.125061 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: Oct 11 11:46:34.128284 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f234c23-e561-408d-9567-ed7b18460f31 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:34.132116 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 131/267] 10.222.0.59 () {56 vars in 1201 bytes} [Tue Oct 11 11:46:34 2022] GET /identity/v3/groups/5e917abb7b794d318682da2487edff33 => generated 226 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:46:34.138918 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303d3c67-4f78-472c-8c4e-6ac5318418b4 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:34.160327 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303d3c67-4f78-472c-8c4e-6ac5318418b4 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:34.168693 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef781fb-41b6-4d5e-8acc-482b02353ea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:34.170358 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef781fb-41b6-4d5e-8acc-482b02353ea1 admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:34.171206 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef781fb-41b6-4d5e-8acc-482b02353ea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:34.171950 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef781fb-41b6-4d5e-8acc-482b02353ea1 admin admin] PATH_INFO: `/v3/projects/1a4b3d97a66e4af4be0a066285f102de` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:34.174492 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef781fb-41b6-4d5e-8acc-482b02353ea1 admin admin] RBAC: Authorizing `identity:get_project(project_id=1a4b3d97a66e4af4be0a066285f102de)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:34.179333 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef781fb-41b6-4d5e-8acc-482b02353ea1 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:34.183247 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 137/268] 10.222.0.59 () {56 vars in 1207 bytes} [Tue Oct 11 11:46:34 2022] GET /identity/v3/projects/1a4b3d97a66e4af4be0a066285f102de => generated 298 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:46:34.190004 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f234c23-e561-408d-9567-ed7b18460f31 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:34.209086 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f234c23-e561-408d-9567-ed7b18460f31 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:34.217624 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc6d8e1-b76d-47f3-a133-d3f386cfe901 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:34.219373 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc6d8e1-b76d-47f3-a133-d3f386cfe901 admin admin] REQUEST_METHOD: `PUT` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:34.220178 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc6d8e1-b76d-47f3-a133-d3f386cfe901 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:34.221070 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc6d8e1-b76d-47f3-a133-d3f386cfe901 admin admin] PATH_INFO: `/v3/projects/1a4b3d97a66e4af4be0a066285f102de/groups/5e917abb7b794d318682da2487edff33/roles/de2d2bd26fff4748a79990f97a107164` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:34.225123 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc6d8e1-b76d-47f3-a133-d3f386cfe901 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a4b3d97a66e4af4be0a066285f102de, group_id=5e917abb7b794d318682da2487edff33, role_id=de2d2bd26fff4748a79990f97a107164)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:34.227591 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: WARNING py.warnings [None req-dcc6d8e1-b76d-47f3-a133-d3f386cfe901 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 11 11:46:34.227591 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: warnings.warn(msg) Oct 11 11:46:34.227591 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: Oct 11 11:46:34.232742 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc6d8e1-b76d-47f3-a133-d3f386cfe901 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:34.263584 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 132/269] 10.222.0.59 () {58 vars in 1463 bytes} [Tue Oct 11 11:46:34 2022] PUT /identity/v3/projects/1a4b3d97a66e4af4be0a066285f102de/groups/5e917abb7b794d318682da2487edff33/roles/de2d2bd26fff4748a79990f97a107164 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 11:46:34.849138 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf782cb-4b06-4e86-a4aa-433545b1b29a None None] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:34.849610 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf782cb-4b06-4e86-a4aa-433545b1b29a None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:34.849770 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf782cb-4b06-4e86-a4aa-433545b1b29a None None] PATH_INFO: `/` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:34.851448 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 138/270] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:46:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:46:34.857437 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965ecf24-74f9-4ffb-bfe9-814607be56d5 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:34.857723 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965ecf24-74f9-4ffb-bfe9-814607be56d5 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:34.858013 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965ecf24-74f9-4ffb-bfe9-814607be56d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:34.888826 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-965ecf24-74f9-4ffb-bfe9-814607be56d5 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:46:34.890065 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-965ecf24-74f9-4ffb-bfe9-814607be56d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:46:34.953179 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414dcb85-5910-4cb7-8af7-3a883cfe2bcf None None] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:34.953427 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414dcb85-5910-4cb7-8af7-3a883cfe2bcf None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:34.953767 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414dcb85-5910-4cb7-8af7-3a883cfe2bcf None None] PATH_INFO: `/` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:34.955119 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 139/271] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:46:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 11 11:46:34.976136 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7c937a-73f1-4fc9-b6e3-f22ec0db8221 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:34.976558 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7c937a-73f1-4fc9-b6e3-f22ec0db8221 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:34.976691 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7c937a-73f1-4fc9-b6e3-f22ec0db8221 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:35.008834 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-be7c937a-73f1-4fc9-b6e3-f22ec0db8221 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:46:35.010106 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-be7c937a-73f1-4fc9-b6e3-f22ec0db8221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:46:35.026600 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 133/272] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:46:34 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Oct 11 11:46:35.142339 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 140/273] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:46:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 11:46:35.508910 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965ecf24-74f9-4ffb-bfe9-814607be56d5 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:35.522698 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965ecf24-74f9-4ffb-bfe9-814607be56d5 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:35.527751 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575e6af2-a9ab-409d-b756-f5cf24d6b2e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:35.528440 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575e6af2-a9ab-409d-b756-f5cf24d6b2e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:35.528722 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575e6af2-a9ab-409d-b756-f5cf24d6b2e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:35.528995 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575e6af2-a9ab-409d-b756-f5cf24d6b2e0 admin admin] PATH_INFO: `/v3/roles/member` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:35.537535 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:35.549770 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575e6af2-a9ab-409d-b756-f5cf24d6b2e0 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 11:46:35.556884 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application [None req-575e6af2-a9ab-409d-b756-f5cf24d6b2e0 admin admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 11 11:46:35.556884 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:46:35.556884 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:46:35.556884 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:46:35.556884 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:46:35.556884 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:46:35.556884 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:46:35.556884 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:46:35.556884 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:46:35.556884 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:46:35.556884 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:46:35.556884 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:46:35.556884 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 11:46:35.556884 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 11:46:35.556884 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 11:46:35.556884 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 11:46:35.558052 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 11:46:35.558052 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 11:46:35.558052 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 11:46:35.558052 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 11:46:35.558052 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:46:35.558052 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:46:35.558052 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:46:35.558052 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:46:35.558052 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:46:35.558052 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:46:35.558052 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:46:35.558052 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application with Lock( Oct 11 11:46:35.558052 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:46:35.558052 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:46:35.558052 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:46:35.558052 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:46:35.559457 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:46:35.559457 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:46:35.559457 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:46:35.559457 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:46:35.559457 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 11:46:35.559457 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 11:46:35.559457 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 11:46:35.559457 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 11:46:35.559457 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 11:46:35.559457 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 11:46:35.559457 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 11 11:46:35.559457 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:46:35.560531 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 134/274] 10.222.0.59 () {56 vars in 1120 bytes} [Tue Oct 11 11:46:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 51 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:46:35.565551 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7c937a-73f1-4fc9-b6e3-f22ec0db8221 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:35.587771 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7c937a-73f1-4fc9-b6e3-f22ec0db8221 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:35.596594 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b058741d-a413-4f55-a96a-b73e387d0bd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:35.597253 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b058741d-a413-4f55-a96a-b73e387d0bd7 admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:35.597533 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b058741d-a413-4f55-a96a-b73e387d0bd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:35.597814 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b058741d-a413-4f55-a96a-b73e387d0bd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:35.598523 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b058741d-a413-4f55-a96a-b73e387d0bd7 admin admin] RBAC: Adding query filter params (name=member) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:46:35.598917 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b058741d-a413-4f55-a96a-b73e387d0bd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:35.600596 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b058741d-a413-4f55-a96a-b73e387d0bd7 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:35.616801 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 141/275] 10.222.0.59 () {56 vars in 1122 bytes} [Tue Oct 11 11:46:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:46:35.623206 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575e6af2-a9ab-409d-b756-f5cf24d6b2e0 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:35.636028 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575e6af2-a9ab-409d-b756-f5cf24d6b2e0 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:35.648597 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ecaef7-69cc-4640-855b-8073caeb5c76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:35.649361 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ecaef7-69cc-4640-855b-8073caeb5c76 admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:35.649581 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ecaef7-69cc-4640-855b-8073caeb5c76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:35.649864 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ecaef7-69cc-4640-855b-8073caeb5c76 admin admin] PATH_INFO: `/v3/groups/5e917abb7b794d318682da2487edff33` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:35.655602 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ecaef7-69cc-4640-855b-8073caeb5c76 admin admin] RBAC: Authorizing `identity:get_group(group_id=5e917abb7b794d318682da2487edff33)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:35.657667 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ecaef7-69cc-4640-855b-8073caeb5c76 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:35.660270 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 135/276] 10.222.0.59 () {56 vars in 1201 bytes} [Tue Oct 11 11:46:35 2022] GET /identity/v3/groups/5e917abb7b794d318682da2487edff33 => generated 226 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:46:35.665706 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b058741d-a413-4f55-a96a-b73e387d0bd7 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:35.680137 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b058741d-a413-4f55-a96a-b73e387d0bd7 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:35.691182 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788d4485-e7a9-4f55-a110-0b839ef6c021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:35.691862 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788d4485-e7a9-4f55-a110-0b839ef6c021 admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:35.692139 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788d4485-e7a9-4f55-a110-0b839ef6c021 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:35.692425 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788d4485-e7a9-4f55-a110-0b839ef6c021 admin admin] PATH_INFO: `/v3/projects/ed233e6a85f94375aece3faac28df210` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:35.693686 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788d4485-e7a9-4f55-a110-0b839ef6c021 admin admin] RBAC: Authorizing `identity:get_project(project_id=ed233e6a85f94375aece3faac28df210)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:35.695876 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788d4485-e7a9-4f55-a110-0b839ef6c021 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:35.697697 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 142/277] 10.222.0.59 () {56 vars in 1207 bytes} [Tue Oct 11 11:46:35 2022] GET /identity/v3/projects/ed233e6a85f94375aece3faac28df210 => generated 302 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:46:35.703969 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ecaef7-69cc-4640-855b-8073caeb5c76 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:35.719323 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ecaef7-69cc-4640-855b-8073caeb5c76 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:35.731357 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf8a6de-1acd-4451-b64f-ced567b198fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:35.732056 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf8a6de-1acd-4451-b64f-ced567b198fd admin admin] REQUEST_METHOD: `PUT` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:35.732338 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf8a6de-1acd-4451-b64f-ced567b198fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:35.732616 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf8a6de-1acd-4451-b64f-ced567b198fd admin admin] PATH_INFO: `/v3/projects/ed233e6a85f94375aece3faac28df210/groups/5e917abb7b794d318682da2487edff33/roles/de2d2bd26fff4748a79990f97a107164` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:35.740518 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf8a6de-1acd-4451-b64f-ced567b198fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed233e6a85f94375aece3faac28df210, group_id=5e917abb7b794d318682da2487edff33, role_id=de2d2bd26fff4748a79990f97a107164)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:35.743468 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf8a6de-1acd-4451-b64f-ced567b198fd admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:35.777604 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 136/278] 10.222.0.59 () {58 vars in 1463 bytes} [Tue Oct 11 11:46:35 2022] PUT /identity/v3/projects/ed233e6a85f94375aece3faac28df210/groups/5e917abb7b794d318682da2487edff33/roles/de2d2bd26fff4748a79990f97a107164 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 11:46:35.916251 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788d4485-e7a9-4f55-a110-0b839ef6c021 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:35.981841 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360e64be-34b9-486c-a71d-126fc9d60eca None None] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:35.982340 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360e64be-34b9-486c-a71d-126fc9d60eca None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:35.982858 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360e64be-34b9-486c-a71d-126fc9d60eca None None] PATH_INFO: `/` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:35.984450 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:35.990781 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 137/279] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:46:35 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:46:35.995634 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4699ff-7432-4ba2-bab2-8193893e6e1b None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:35.996107 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4699ff-7432-4ba2-bab2-8193893e6e1b None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:35.996578 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4699ff-7432-4ba2-bab2-8193893e6e1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:36.052524 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-9e4699ff-7432-4ba2-bab2-8193893e6e1b None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:46:36.053577 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-9e4699ff-7432-4ba2-bab2-8193893e6e1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:46:36.137870 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788d4485-e7a9-4f55-a110-0b839ef6c021 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:36.206711 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed2198d-0ca6-46f5-8712-93688f46785f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:36.207749 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed2198d-0ca6-46f5-8712-93688f46785f None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:36.207749 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed2198d-0ca6-46f5-8712-93688f46785f None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:36.208010 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed2198d-0ca6-46f5-8712-93688f46785f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:36.210116 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 138/280] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:46:35 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Oct 11 11:46:36.220422 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed2198d-0ca6-46f5-8712-93688f46785f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 11:46:36.225619 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application [None req-bed2198d-0ca6-46f5-8712-93688f46785f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 11 11:46:36.225619 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:46:36.225619 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:46:36.225619 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:46:36.225619 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:46:36.225619 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:46:36.225619 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:46:36.225619 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:46:36.225619 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:46:36.225619 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:46:36.225619 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:46:36.225619 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:46:36.225619 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 11:46:36.225619 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 11:46:36.225619 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 11:46:36.225619 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 11:46:36.227305 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 11:46:36.227305 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 11:46:36.227305 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 11:46:36.227305 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 11:46:36.227305 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:46:36.227305 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:46:36.227305 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:46:36.227305 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:46:36.227305 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:46:36.227305 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:46:36.227305 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:46:36.227305 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application with Lock( Oct 11 11:46:36.227305 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:46:36.227305 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:46:36.227305 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:46:36.227305 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:46:36.228658 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:46:36.228658 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:46:36.228658 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:46:36.228658 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:46:36.228658 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 11:46:36.228658 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 11:46:36.228658 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 11:46:36.228658 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 11:46:36.228658 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 11:46:36.228658 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 11:46:36.228658 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 11 11:46:36.228658 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:46:36.228658 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 143/281] 10.222.0.59 () {56 vars in 1099 bytes} [Tue Oct 11 11:46:35 2022] GET /identity/v3/roles/reader => generated 84 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:46:36.233290 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4699ff-7432-4ba2-bab2-8193893e6e1b None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:36.364026 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4699ff-7432-4ba2-bab2-8193893e6e1b None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:36.395850 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:36.409514 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b136e8cc-ec83-4919-9438-b5f9a50e4c54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:36.411401 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b136e8cc-ec83-4919-9438-b5f9a50e4c54 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:36.411689 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b136e8cc-ec83-4919-9438-b5f9a50e4c54 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:36.411952 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b136e8cc-ec83-4919-9438-b5f9a50e4c54 None admin] PATH_INFO: `/v3/roles` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:36.413093 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b136e8cc-ec83-4919-9438-b5f9a50e4c54 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:46:36.414008 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b136e8cc-ec83-4919-9438-b5f9a50e4c54 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:36.416707 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b136e8cc-ec83-4919-9438-b5f9a50e4c54 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:36.435745 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 139/282] 10.222.0.59 () {56 vars in 1101 bytes} [Tue Oct 11 11:46:36 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:46:36.441790 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed2198d-0ca6-46f5-8712-93688f46785f None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:36.556394 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:36.568054 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed2198d-0ca6-46f5-8712-93688f46785f None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:36.609058 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc34b0d-feb6-4f7c-abf2-4faadf338e33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:36.609707 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc34b0d-feb6-4f7c-abf2-4faadf338e33 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:36.609985 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc34b0d-feb6-4f7c-abf2-4faadf338e33 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:36.610243 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc34b0d-feb6-4f7c-abf2-4faadf338e33 None admin] PATH_INFO: `/v3/users/0a837f6cfa85408a8a466d293da5b5b2` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:36.611450 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc34b0d-feb6-4f7c-abf2-4faadf338e33 None admin] RBAC: Authorizing `identity:get_user(user_id=0a837f6cfa85408a8a466d293da5b5b2)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:36.612972 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc34b0d-feb6-4f7c-abf2-4faadf338e33 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:36.615088 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 144/283] 10.222.0.59 () {56 vars in 1177 bytes} [Tue Oct 11 11:46:36 2022] GET /identity/v3/users/0a837f6cfa85408a8a466d293da5b5b2 => generated 293 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:46:36.621021 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b136e8cc-ec83-4919-9438-b5f9a50e4c54 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:36.642231 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926c1c48-c286-4db1-bf0a-9b09f6e2031b None None] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:36.642532 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926c1c48-c286-4db1-bf0a-9b09f6e2031b None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:36.642718 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926c1c48-c286-4db1-bf0a-9b09f6e2031b None None] PATH_INFO: `/` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:36.644110 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 145/284] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:46:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:46:36.652315 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d858cde-9f3a-4a2f-9fe3-741e0762e0ed None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:36.652755 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d858cde-9f3a-4a2f-9fe3-741e0762e0ed None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:36.653171 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d858cde-9f3a-4a2f-9fe3-741e0762e0ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:36.675989 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-5d858cde-9f3a-4a2f-9fe3-741e0762e0ed None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:46:36.676807 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-5d858cde-9f3a-4a2f-9fe3-741e0762e0ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:46:36.689257 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 146/285] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:46:36 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Oct 11 11:46:36.792347 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d858cde-9f3a-4a2f-9fe3-741e0762e0ed None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:36.812060 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d858cde-9f3a-4a2f-9fe3-741e0762e0ed None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:36.821258 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b136e8cc-ec83-4919-9438-b5f9a50e4c54 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:36.822429 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145d2f36-dbef-4b56-bc13-d5b8ddaf5008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:36.823240 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145d2f36-dbef-4b56-bc13-d5b8ddaf5008 admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:36.823582 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145d2f36-dbef-4b56-bc13-d5b8ddaf5008 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:36.823914 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145d2f36-dbef-4b56-bc13-d5b8ddaf5008 admin admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:36.842921 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145d2f36-dbef-4b56-bc13-d5b8ddaf5008 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 11:46:36.852276 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application [None req-145d2f36-dbef-4b56-bc13-d5b8ddaf5008 admin admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 11 11:46:36.852276 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:46:36.852276 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:46:36.852276 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:46:36.852276 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:46:36.852276 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:46:36.852276 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:46:36.852276 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:46:36.852276 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:46:36.852276 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:46:36.852276 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:46:36.852276 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:46:36.852276 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 11:46:36.852276 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 11:46:36.852276 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 11:46:36.852276 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 11:46:36.854004 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 11:46:36.854004 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 11:46:36.854004 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 11:46:36.854004 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 11:46:36.854004 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:46:36.854004 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:46:36.854004 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:46:36.854004 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:46:36.854004 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:46:36.854004 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:46:36.854004 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:46:36.854004 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application with Lock( Oct 11 11:46:36.854004 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:46:36.854004 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:46:36.854004 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:46:36.854004 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:46:36.855534 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:46:36.855534 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:46:36.855534 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:46:36.855534 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:46:36.855534 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 11:46:36.855534 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 11:46:36.855534 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 11:46:36.855534 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 11:46:36.855534 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 11:46:36.855534 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 11:46:36.855534 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 11 11:46:36.855534 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:46:36.855534 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 147/286] 10.222.0.59 () {56 vars in 1135 bytes} [Tue Oct 11 11:46:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 62 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 11 11:46:36.862530 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145d2f36-dbef-4b56-bc13-d5b8ddaf5008 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:36.883612 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145d2f36-dbef-4b56-bc13-d5b8ddaf5008 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:36.903386 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e07beae-6148-4cf5-a11c-403770cc26f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:36.903999 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e07beae-6148-4cf5-a11c-403770cc26f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:36.904283 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e07beae-6148-4cf5-a11c-403770cc26f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:36.904551 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e07beae-6148-4cf5-a11c-403770cc26f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:36.905238 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e07beae-6148-4cf5-a11c-403770cc26f6 admin admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:46:36.905540 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e07beae-6148-4cf5-a11c-403770cc26f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:36.907235 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e07beae-6148-4cf5-a11c-403770cc26f6 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:36.907669 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa203a7-f5df-426e-afa3-be0293774560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:36.908326 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa203a7-f5df-426e-afa3-be0293774560 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:36.908600 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa203a7-f5df-426e-afa3-be0293774560 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:36.908923 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa203a7-f5df-426e-afa3-be0293774560 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:36.910107 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa203a7-f5df-426e-afa3-be0293774560 None admin] RBAC: Adding query filter params (role.id=acda9dcd3a754265af99c2abc579d2da, user.id=0a837f6cfa85408a8a466d293da5b5b2, scope.system=all) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:46:36.910408 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa203a7-f5df-426e-afa3-be0293774560 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:36.916118 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa203a7-f5df-426e-afa3-be0293774560 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:36.924177 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 148/287] 10.222.0.59 () {56 vars in 1132 bytes} [Tue Oct 11 11:46:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:46:36.930883 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e07beae-6148-4cf5-a11c-403770cc26f6 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:36.953675 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 140/288] 10.222.0.59 () {56 vars in 1308 bytes} [Tue Oct 11 11:46:36 2022] GET /identity/v3/role_assignments?user.id=0a837f6cfa85408a8a466d293da5b5b2&scope.system=all&role.id=acda9dcd3a754265af99c2abc579d2da => generated 510 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 11:46:36.958346 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e07beae-6148-4cf5-a11c-403770cc26f6 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:36.977264 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d05d584-d9d8-419f-b48b-c38572aa8e69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:36.977944 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d05d584-d9d8-419f-b48b-c38572aa8e69 admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:36.978218 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d05d584-d9d8-419f-b48b-c38572aa8e69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:36.978484 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d05d584-d9d8-419f-b48b-c38572aa8e69 admin admin] PATH_INFO: `/v3/groups/5e917abb7b794d318682da2487edff33` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:36.983053 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d05d584-d9d8-419f-b48b-c38572aa8e69 admin admin] RBAC: Authorizing `identity:get_group(group_id=5e917abb7b794d318682da2487edff33)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:36.985050 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d05d584-d9d8-419f-b48b-c38572aa8e69 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:36.987213 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 149/289] 10.222.0.59 () {56 vars in 1201 bytes} [Tue Oct 11 11:46:36 2022] GET /identity/v3/groups/5e917abb7b794d318682da2487edff33 => generated 226 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:46:36.993600 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa203a7-f5df-426e-afa3-be0293774560 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:37.020034 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa203a7-f5df-426e-afa3-be0293774560 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:37.032060 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406d2c8b-8e61-42fb-a84b-6585ff247ceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:37.033969 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406d2c8b-8e61-42fb-a84b-6585ff247ceb admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:37.039434 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406d2c8b-8e61-42fb-a84b-6585ff247ceb admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:37.040489 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406d2c8b-8e61-42fb-a84b-6585ff247ceb admin admin] PATH_INFO: `/v3/projects/ed233e6a85f94375aece3faac28df210` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:37.043503 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406d2c8b-8e61-42fb-a84b-6585ff247ceb admin admin] RBAC: Authorizing `identity:get_project(project_id=ed233e6a85f94375aece3faac28df210)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:37.048188 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406d2c8b-8e61-42fb-a84b-6585ff247ceb admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:37.052419 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 141/290] 10.222.0.59 () {56 vars in 1207 bytes} [Tue Oct 11 11:46:36 2022] GET /identity/v3/projects/ed233e6a85f94375aece3faac28df210 => generated 302 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:46:37.058488 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d05d584-d9d8-419f-b48b-c38572aa8e69 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:37.077348 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d05d584-d9d8-419f-b48b-c38572aa8e69 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:37.092040 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99faaa35-5f98-4dd0-95f7-6b6f269bbb77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:37.092740 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99faaa35-5f98-4dd0-95f7-6b6f269bbb77 admin admin] REQUEST_METHOD: `PUT` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:37.093036 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99faaa35-5f98-4dd0-95f7-6b6f269bbb77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:37.093296 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99faaa35-5f98-4dd0-95f7-6b6f269bbb77 admin admin] PATH_INFO: `/v3/projects/ed233e6a85f94375aece3faac28df210/groups/5e917abb7b794d318682da2487edff33/roles/3a1a5476791547419a0b885c2022ef68` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:37.100813 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99faaa35-5f98-4dd0-95f7-6b6f269bbb77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed233e6a85f94375aece3faac28df210, group_id=5e917abb7b794d318682da2487edff33, role_id=3a1a5476791547419a0b885c2022ef68)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:37.103602 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99faaa35-5f98-4dd0-95f7-6b6f269bbb77 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:37.133183 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 150/291] 10.222.0.59 () {58 vars in 1463 bytes} [Tue Oct 11 11:46:37 2022] PUT /identity/v3/projects/ed233e6a85f94375aece3faac28df210/groups/5e917abb7b794d318682da2487edff33/roles/3a1a5476791547419a0b885c2022ef68 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 11:46:37.276951 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406d2c8b-8e61-42fb-a84b-6585ff247ceb admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:37.297181 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406d2c8b-8e61-42fb-a84b-6585ff247ceb admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:37.402972 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:37.431881 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d819bd5-44cf-416e-81f1-122e8bb395bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:37.433020 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d819bd5-44cf-416e-81f1-122e8bb395bd admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:37.433499 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d819bd5-44cf-416e-81f1-122e8bb395bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:37.434002 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d819bd5-44cf-416e-81f1-122e8bb395bd admin admin] PATH_INFO: `/v3/roles/admin` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:37.459232 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d819bd5-44cf-416e-81f1-122e8bb395bd admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 11:46:37.471181 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application [None req-8d819bd5-44cf-416e-81f1-122e8bb395bd admin admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 11 11:46:37.471181 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:46:37.471181 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:46:37.471181 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:46:37.471181 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:46:37.471181 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:46:37.471181 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:46:37.471181 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:46:37.471181 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:46:37.471181 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:46:37.471181 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:46:37.471181 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:46:37.471181 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 11:46:37.471181 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 11:46:37.471181 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 11:46:37.471181 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 11:46:37.472210 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 11:46:37.472210 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 11:46:37.472210 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 11:46:37.472210 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 11:46:37.472210 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:46:37.472210 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:46:37.472210 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:46:37.472210 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:46:37.472210 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:46:37.472210 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:46:37.472210 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:46:37.472210 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application with Lock( Oct 11 11:46:37.472210 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:46:37.472210 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:46:37.472210 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:46:37.472210 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:46:37.472999 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:46:37.472999 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:46:37.472999 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:46:37.472999 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:46:37.472999 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 11:46:37.472999 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 11:46:37.472999 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 11:46:37.472999 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 11:46:37.472999 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 11:46:37.472999 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 11:46:37.472999 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 11 11:46:37.472999 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:46:37.474174 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 142/292] 10.222.0.59 () {56 vars in 1117 bytes} [Tue Oct 11 11:46:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:46:37.479657 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99faaa35-5f98-4dd0-95f7-6b6f269bbb77 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:37.503180 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99faaa35-5f98-4dd0-95f7-6b6f269bbb77 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:37.516573 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a47c80-54b5-481f-9632-5ee4fc4247a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:37.517214 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a47c80-54b5-481f-9632-5ee4fc4247a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:37.517493 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a47c80-54b5-481f-9632-5ee4fc4247a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:37.517765 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a47c80-54b5-481f-9632-5ee4fc4247a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:37.518502 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a47c80-54b5-481f-9632-5ee4fc4247a8 admin admin] RBAC: Adding query filter params (name=admin) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:46:37.520361 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a47c80-54b5-481f-9632-5ee4fc4247a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:37.520672 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a47c80-54b5-481f-9632-5ee4fc4247a8 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:37.539598 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 151/293] 10.222.0.59 () {56 vars in 1120 bytes} [Tue Oct 11 11:46:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:46:37.546179 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d819bd5-44cf-416e-81f1-122e8bb395bd admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:37.557761 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d819bd5-44cf-416e-81f1-122e8bb395bd admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:37.566393 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1669244c-0dae-4a90-b4d0-b8307946a188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:37.567386 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1669244c-0dae-4a90-b4d0-b8307946a188 admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:37.567843 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1669244c-0dae-4a90-b4d0-b8307946a188 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:37.568255 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1669244c-0dae-4a90-b4d0-b8307946a188 admin admin] PATH_INFO: `/v3/groups/61c402f2dab14543932c242c96eb8782` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:37.576694 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1669244c-0dae-4a90-b4d0-b8307946a188 admin admin] RBAC: Authorizing `identity:get_group(group_id=61c402f2dab14543932c242c96eb8782)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:37.579122 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1669244c-0dae-4a90-b4d0-b8307946a188 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:37.581256 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 143/294] 10.222.0.59 () {56 vars in 1201 bytes} [Tue Oct 11 11:46:37 2022] GET /identity/v3/groups/61c402f2dab14543932c242c96eb8782 => generated 229 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:46:37.588432 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a47c80-54b5-481f-9632-5ee4fc4247a8 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:37.617589 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a47c80-54b5-481f-9632-5ee4fc4247a8 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:37.629815 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc22cc6-906e-437d-b326-989e993440b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:37.631012 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc22cc6-906e-437d-b326-989e993440b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:37.631453 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc22cc6-906e-437d-b326-989e993440b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:37.631914 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc22cc6-906e-437d-b326-989e993440b2 admin admin] PATH_INFO: `/v3/projects/ec9b5254352a410fb1f7cca5a5393294` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:37.637649 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc22cc6-906e-437d-b326-989e993440b2 admin admin] RBAC: Authorizing `identity:get_project(project_id=ec9b5254352a410fb1f7cca5a5393294)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:37.641083 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc22cc6-906e-437d-b326-989e993440b2 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:37.644993 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 152/295] 10.222.0.59 () {56 vars in 1207 bytes} [Tue Oct 11 11:46:37 2022] GET /identity/v3/projects/ec9b5254352a410fb1f7cca5a5393294 => generated 344 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 11:46:37.650882 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1669244c-0dae-4a90-b4d0-b8307946a188 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:37.665967 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1669244c-0dae-4a90-b4d0-b8307946a188 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:37.672591 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c8c011-9c61-42e0-82b6-99c32028f8c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:37.673602 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c8c011-9c61-42e0-82b6-99c32028f8c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:37.674025 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c8c011-9c61-42e0-82b6-99c32028f8c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:37.674524 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c8c011-9c61-42e0-82b6-99c32028f8c5 admin admin] PATH_INFO: `/v3/projects/ec9b5254352a410fb1f7cca5a5393294/groups/61c402f2dab14543932c242c96eb8782/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:37.676956 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c8c011-9c61-42e0-82b6-99c32028f8c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec9b5254352a410fb1f7cca5a5393294, group_id=61c402f2dab14543932c242c96eb8782, role_id=5cf1f4bd2fef46ab9c6eff56d52f9f69)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:37.680072 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c8c011-9c61-42e0-82b6-99c32028f8c5 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:37.700344 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 144/296] 10.222.0.59 () {58 vars in 1463 bytes} [Tue Oct 11 11:46:37 2022] PUT /identity/v3/projects/ec9b5254352a410fb1f7cca5a5393294/groups/61c402f2dab14543932c242c96eb8782/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 11:46:37.942357 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdfd8fe-a231-4723-9cc8-c159105edcb4 None None] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:37.942907 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdfd8fe-a231-4723-9cc8-c159105edcb4 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:37.943325 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdfd8fe-a231-4723-9cc8-c159105edcb4 None None] PATH_INFO: `/` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:37.945179 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 153/297] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:46:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:46:37.950917 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444207a2-ee12-45aa-bef4-caa7dfc6db68 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:37.951225 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444207a2-ee12-45aa-bef4-caa7dfc6db68 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:37.951487 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444207a2-ee12-45aa-bef4-caa7dfc6db68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:37.979118 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-444207a2-ee12-45aa-bef4-caa7dfc6db68 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:46:37.980345 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-444207a2-ee12-45aa-bef4-caa7dfc6db68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:46:38.073155 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 145/298] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:46:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Oct 11 11:46:38.186812 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2880dd-d9d4-49c1-862b-aa8d6ab8622c None None] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:38.187099 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2880dd-d9d4-49c1-862b-aa8d6ab8622c None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:38.187379 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2880dd-d9d4-49c1-862b-aa8d6ab8622c None None] PATH_INFO: `/` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:38.191677 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 154/299] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:46:38 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:46:38.197773 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b975bc-068c-46b3-a168-036c482e495e None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:38.198384 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b975bc-068c-46b3-a168-036c482e495e None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:38.198880 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b975bc-068c-46b3-a168-036c482e495e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:38.223113 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-22b975bc-068c-46b3-a168-036c482e495e None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:46:38.224695 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-22b975bc-068c-46b3-a168-036c482e495e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:46:38.332738 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 146/300] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:46:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 11:46:38.459028 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2880dd-d9d4-49c1-862b-aa8d6ab8622c None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:38.596662 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2880dd-d9d4-49c1-862b-aa8d6ab8622c None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:38.599442 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:38.643538 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bd8d73-3b69-4393-99dd-cf4d8263bece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:38.644225 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bd8d73-3b69-4393-99dd-cf4d8263bece None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:38.644507 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bd8d73-3b69-4393-99dd-cf4d8263bece None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:38.644782 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bd8d73-3b69-4393-99dd-cf4d8263bece None admin] PATH_INFO: `/v3/roles/member` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:38.657403 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bd8d73-3b69-4393-99dd-cf4d8263bece None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 11:46:38.662822 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application [None req-69bd8d73-3b69-4393-99dd-cf4d8263bece None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 11 11:46:38.662822 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:46:38.662822 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:46:38.662822 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:46:38.662822 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:46:38.662822 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:46:38.662822 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:46:38.662822 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:46:38.662822 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:46:38.662822 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:46:38.662822 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:46:38.662822 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:46:38.662822 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 11:46:38.662822 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 11:46:38.662822 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 11:46:38.662822 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 11:46:38.663757 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 11:46:38.663757 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 11:46:38.663757 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 11:46:38.663757 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 11:46:38.663757 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:46:38.663757 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:46:38.663757 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:46:38.663757 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:46:38.663757 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:46:38.663757 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:46:38.663757 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:46:38.663757 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application with Lock( Oct 11 11:46:38.663757 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:46:38.663757 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:46:38.663757 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:46:38.663757 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:46:38.664563 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:46:38.664563 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:46:38.664563 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:46:38.664563 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:46:38.664563 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 11:46:38.664563 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 11:46:38.664563 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 11:46:38.664563 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 11:46:38.664563 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 11:46:38.664563 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 11:46:38.664563 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 11 11:46:38.664563 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:46:38.664563 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 155/301] 10.222.0.59 () {56 vars in 1099 bytes} [Tue Oct 11 11:46:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:46:38.670345 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b975bc-068c-46b3-a168-036c482e495e None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:38.710002 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:38.807829 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b975bc-068c-46b3-a168-036c482e495e None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:38.842946 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bd8d73-3b69-4393-99dd-cf4d8263bece None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:38.852725 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0065c0b-5215-4fae-9767-2ae25c527528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:38.853734 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0065c0b-5215-4fae-9767-2ae25c527528 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:38.854209 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0065c0b-5215-4fae-9767-2ae25c527528 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:38.854699 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0065c0b-5215-4fae-9767-2ae25c527528 None admin] PATH_INFO: `/v3/roles` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:38.855690 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0065c0b-5215-4fae-9767-2ae25c527528 None admin] RBAC: Adding query filter params (name=member) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:46:38.856419 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0065c0b-5215-4fae-9767-2ae25c527528 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:38.862654 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0065c0b-5215-4fae-9767-2ae25c527528 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:38.878943 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 147/302] 10.222.0.59 () {56 vars in 1101 bytes} [Tue Oct 11 11:46:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:46:38.886366 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0065c0b-5215-4fae-9767-2ae25c527528 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:39.095014 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:39.107515 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bd8d73-3b69-4393-99dd-cf4d8263bece None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:39.135500 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0065c0b-5215-4fae-9767-2ae25c527528 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:39.184480 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2455b381-ad15-4859-a2cf-bfe12396a5a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:39.185135 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2455b381-ad15-4859-a2cf-bfe12396a5a3 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:39.185408 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2455b381-ad15-4859-a2cf-bfe12396a5a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:39.185683 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2455b381-ad15-4859-a2cf-bfe12396a5a3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:39.208041 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2455b381-ad15-4859-a2cf-bfe12396a5a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 11:46:39.214214 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc35c52-0b06-44ea-910f-0177706800b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:39.214887 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc35c52-0b06-44ea-910f-0177706800b9 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:39.215162 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc35c52-0b06-44ea-910f-0177706800b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:39.215434 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc35c52-0b06-44ea-910f-0177706800b9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:39.216800 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application [None req-2455b381-ad15-4859-a2cf-bfe12396a5a3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 11 11:46:39.216800 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:46:39.216800 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:46:39.216800 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:46:39.216800 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:46:39.216800 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:46:39.216800 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:46:39.216800 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:46:39.216800 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:46:39.216800 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:46:39.216800 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:46:39.216800 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:46:39.216800 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 11:46:39.216800 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 11:46:39.216800 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 11:46:39.216800 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 11:46:39.217758 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 11:46:39.217758 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 11:46:39.217758 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 11:46:39.217758 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 11:46:39.217758 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:46:39.217758 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:46:39.217758 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:46:39.217758 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:46:39.217758 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:46:39.217758 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:46:39.217758 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:46:39.217758 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application with Lock( Oct 11 11:46:39.217758 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:46:39.217758 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:46:39.217758 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:46:39.217758 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:46:39.218578 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:46:39.218578 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:46:39.218578 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:46:39.218578 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:46:39.218578 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 11:46:39.218578 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 11:46:39.218578 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 11:46:39.218578 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 11:46:39.218578 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 11:46:39.218578 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 11:46:39.218578 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 11 11:46:39.218578 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:46:39.221375 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 156/303] 10.222.0.59 () {56 vars in 1114 bytes} [Tue Oct 11 11:46:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 378 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:46:39.227755 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2455b381-ad15-4859-a2cf-bfe12396a5a3 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:39.285713 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:39.337865 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc35c52-0b06-44ea-910f-0177706800b9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:39.340758 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc35c52-0b06-44ea-910f-0177706800b9 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:39.459890 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:39.471741 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 148/304] 10.222.0.59 () {58 vars in 1282 bytes} [Tue Oct 11 11:46:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 586 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 11 11:46:39.480041 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc35c52-0b06-44ea-910f-0177706800b9 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:39.508769 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2455b381-ad15-4859-a2cf-bfe12396a5a3 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:39.600034 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0009b8fa-e0f0-4c44-b7ab-6b4cf634a8dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:39.601005 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0009b8fa-e0f0-4c44-b7ab-6b4cf634a8dd None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:39.601005 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0009b8fa-e0f0-4c44-b7ab-6b4cf634a8dd None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:39.601264 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0009b8fa-e0f0-4c44-b7ab-6b4cf634a8dd None admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:39.601890 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0009b8fa-e0f0-4c44-b7ab-6b4cf634a8dd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:46:39.602192 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0009b8fa-e0f0-4c44-b7ab-6b4cf634a8dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:39.603704 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0009b8fa-e0f0-4c44-b7ab-6b4cf634a8dd None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:39.609874 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc35c52-0b06-44ea-910f-0177706800b9 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:39.625013 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 157/305] 10.222.0.59 () {56 vars in 1111 bytes} [Tue Oct 11 11:46:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 398 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:46:39.631428 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0009b8fa-e0f0-4c44-b7ab-6b4cf634a8dd None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:39.654118 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a33adaa-aebb-46e5-9fd3-2f3a4587f5b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:39.654818 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a33adaa-aebb-46e5-9fd3-2f3a4587f5b8 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:39.655108 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a33adaa-aebb-46e5-9fd3-2f3a4587f5b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:39.655375 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a33adaa-aebb-46e5-9fd3-2f3a4587f5b8 None admin] PATH_INFO: `/v3/projects/1a4b3d97a66e4af4be0a066285f102de` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:39.656597 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a33adaa-aebb-46e5-9fd3-2f3a4587f5b8 None admin] RBAC: Authorizing `identity:get_project(project_id=1a4b3d97a66e4af4be0a066285f102de)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:39.658717 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a33adaa-aebb-46e5-9fd3-2f3a4587f5b8 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:39.661561 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 149/306] 10.222.0.59 () {56 vars in 1186 bytes} [Tue Oct 11 11:46:39 2022] GET /identity/v3/projects/1a4b3d97a66e4af4be0a066285f102de => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:46:39.667624 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a33adaa-aebb-46e5-9fd3-2f3a4587f5b8 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:39.734097 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:39.794613 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a33adaa-aebb-46e5-9fd3-2f3a4587f5b8 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:39.836138 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7fe4ef-0097-4574-a2f8-897cd9487b59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:39.836860 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7fe4ef-0097-4574-a2f8-897cd9487b59 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:39.837138 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7fe4ef-0097-4574-a2f8-897cd9487b59 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:39.837450 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7fe4ef-0097-4574-a2f8-897cd9487b59 None admin] PATH_INFO: `/v3/groups/5e917abb7b794d318682da2487edff33` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:39.838642 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7fe4ef-0097-4574-a2f8-897cd9487b59 None admin] RBAC: Authorizing `identity:get_group(group_id=5e917abb7b794d318682da2487edff33)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:39.840086 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7fe4ef-0097-4574-a2f8-897cd9487b59 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:39.846603 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 150/307] 10.222.0.59 () {56 vars in 1180 bytes} [Tue Oct 11 11:46:39 2022] GET /identity/v3/groups/5e917abb7b794d318682da2487edff33 => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:46:39.848193 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7fe4ef-0097-4574-a2f8-897cd9487b59 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:39.916506 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:39.980639 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0009b8fa-e0f0-4c44-b7ab-6b4cf634a8dd None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:39.982419 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7fe4ef-0097-4574-a2f8-897cd9487b59 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:40.007031 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:40.025519 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3a9946-edc2-40d3-943c-284703c7601a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:40.026444 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3a9946-edc2-40d3-943c-284703c7601a None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:40.027112 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3a9946-edc2-40d3-943c-284703c7601a None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:40.027523 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3a9946-edc2-40d3-943c-284703c7601a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:40.028405 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3a9946-edc2-40d3-943c-284703c7601a None admin] RBAC: Adding query filter params (group.id=5e917abb7b794d318682da2487edff33, role.id=de2d2bd26fff4748a79990f97a107164, scope.project.id=1a4b3d97a66e4af4be0a066285f102de) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:46:40.028911 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3a9946-edc2-40d3-943c-284703c7601a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:40.030635 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3a9946-edc2-40d3-943c-284703c7601a None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:40.038458 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 151/308] 10.222.0.59 () {56 vars in 1376 bytes} [Tue Oct 11 11:46:39 2022] GET /identity/v3/role_assignments?group.id=5e917abb7b794d318682da2487edff33&scope.project.id=1a4b3d97a66e4af4be0a066285f102de&role.id=de2d2bd26fff4748a79990f97a107164 => generated 611 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 11:46:40.079173 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf9a111-0ac5-4445-97c4-e464dae56698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:40.079840 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf9a111-0ac5-4445-97c4-e464dae56698 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:40.080119 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf9a111-0ac5-4445-97c4-e464dae56698 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:40.080393 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf9a111-0ac5-4445-97c4-e464dae56698 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:40.189808 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf9a111-0ac5-4445-97c4-e464dae56698 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:40.191430 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf9a111-0ac5-4445-97c4-e464dae56698 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:40.227625 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f854a1-5492-4945-a034-b2f04f175105 None None] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:40.228495 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f854a1-5492-4945-a034-b2f04f175105 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:40.229506 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f854a1-5492-4945-a034-b2f04f175105 None None] PATH_INFO: `/` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:40.234488 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 152/309] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:46:40 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 11 11:46:40.240819 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717b2e88-6634-4815-80fe-7dc69a2abb4b None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:40.241449 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717b2e88-6634-4815-80fe-7dc69a2abb4b None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:40.241917 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717b2e88-6634-4815-80fe-7dc69a2abb4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:40.271302 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-717b2e88-6634-4815-80fe-7dc69a2abb4b None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:46:40.272995 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-717b2e88-6634-4815-80fe-7dc69a2abb4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:46:40.344321 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:40.369041 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 153/310] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:46:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 11 11:46:40.375554 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 158/311] 10.222.0.59 () {58 vars in 1282 bytes} [Tue Oct 11 11:46:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 745 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 11 11:46:40.381584 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717b2e88-6634-4815-80fe-7dc69a2abb4b None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:40.387074 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e704a7e7-a74a-4225-95f3-7df53944eb7e None None] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:40.387419 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e704a7e7-a74a-4225-95f3-7df53944eb7e None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:40.387631 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e704a7e7-a74a-4225-95f3-7df53944eb7e None None] PATH_INFO: `/` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:40.389564 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 159/312] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:46:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:46:40.397450 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052fb58a-c6bf-4a72-8482-e3dd25812023 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:40.397797 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052fb58a-c6bf-4a72-8482-e3dd25812023 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:40.398039 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052fb58a-c6bf-4a72-8482-e3dd25812023 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:40.429951 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-052fb58a-c6bf-4a72-8482-e3dd25812023 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:46:40.431230 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-052fb58a-c6bf-4a72-8482-e3dd25812023 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:46:40.528051 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717b2e88-6634-4815-80fe-7dc69a2abb4b None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:40.546531 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:40.574435 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 160/313] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:46:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 11 11:46:40.588037 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8035bc43-41ef-4201-baf0-0dc742d32469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:40.589380 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8035bc43-41ef-4201-baf0-0dc742d32469 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:40.590101 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8035bc43-41ef-4201-baf0-0dc742d32469 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:40.590862 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8035bc43-41ef-4201-baf0-0dc742d32469 None admin] PATH_INFO: `/v3/projects/1a4b3d97a66e4af4be0a066285f102de` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:40.593178 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8035bc43-41ef-4201-baf0-0dc742d32469 None admin] RBAC: Authorizing `identity:get_project(project_id=1a4b3d97a66e4af4be0a066285f102de)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:40.595548 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8035bc43-41ef-4201-baf0-0dc742d32469 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:40.599123 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 154/314] 10.222.0.59 () {56 vars in 1186 bytes} [Tue Oct 11 11:46:40 2022] GET /identity/v3/projects/1a4b3d97a66e4af4be0a066285f102de => generated 298 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:46:40.607628 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052fb58a-c6bf-4a72-8482-e3dd25812023 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:40.709180 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8035bc43-41ef-4201-baf0-0dc742d32469 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:40.809020 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052fb58a-c6bf-4a72-8482-e3dd25812023 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:40.835990 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:40.871085 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce5ed9c-bb47-4479-96cd-1854418294cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:40.871767 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce5ed9c-bb47-4479-96cd-1854418294cd None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:40.872594 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce5ed9c-bb47-4479-96cd-1854418294cd None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:40.872594 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce5ed9c-bb47-4479-96cd-1854418294cd None admin] PATH_INFO: `/v3/groups/5e917abb7b794d318682da2487edff33` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:40.873794 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce5ed9c-bb47-4479-96cd-1854418294cd None admin] RBAC: Authorizing `identity:get_group(group_id=5e917abb7b794d318682da2487edff33)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:40.875378 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce5ed9c-bb47-4479-96cd-1854418294cd None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:40.877154 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 161/315] 10.222.0.59 () {56 vars in 1180 bytes} [Tue Oct 11 11:46:40 2022] GET /identity/v3/groups/5e917abb7b794d318682da2487edff33 => generated 226 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:46:40.879341 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b419120b-8333-43f0-87be-c213f90f8834 None None] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:40.879341 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b419120b-8333-43f0-87be-c213f90f8834 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:40.879341 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b419120b-8333-43f0-87be-c213f90f8834 None None] PATH_INFO: `/` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:40.880790 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 162/316] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:46:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:46:40.885854 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b419120b-8333-43f0-87be-c213f90f8834 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:40.891876 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8035bc43-41ef-4201-baf0-0dc742d32469 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:40.943967 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faca31fd-1171-46b2-b2a6-42967680cf99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:40.944769 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faca31fd-1171-46b2-b2a6-42967680cf99 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:40.946174 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faca31fd-1171-46b2-b2a6-42967680cf99 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:40.946174 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faca31fd-1171-46b2-b2a6-42967680cf99 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:40.962639 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faca31fd-1171-46b2-b2a6-42967680cf99 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 11:46:40.967995 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application [None req-faca31fd-1171-46b2-b2a6-42967680cf99 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 11 11:46:40.967995 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:46:40.967995 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:46:40.967995 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:46:40.967995 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:46:40.967995 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:46:40.967995 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:46:40.967995 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:46:40.967995 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:46:40.967995 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:46:40.967995 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:46:40.967995 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:46:40.967995 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 11:46:40.967995 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 11:46:40.967995 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 11:46:40.967995 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 11:46:40.968907 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 11:46:40.968907 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 11:46:40.968907 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 11:46:40.968907 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 11:46:40.968907 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:46:40.968907 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:46:40.968907 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:46:40.968907 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:46:40.968907 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:46:40.968907 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:46:40.968907 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:46:40.968907 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application with Lock( Oct 11 11:46:40.968907 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:46:40.968907 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:46:40.968907 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:46:40.968907 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:46:40.969698 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:46:40.969698 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:46:40.969698 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:46:40.969698 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:46:40.969698 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 11:46:40.969698 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 11:46:40.969698 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 11:46:40.969698 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 11:46:40.969698 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 11:46:40.969698 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 11:46:40.969698 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 11 11:46:40.969698 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:46:40.970847 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 155/317] 10.222.0.59 () {56 vars in 1114 bytes} [Tue Oct 11 11:46:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:46:40.972080 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4940aa18-f7a9-4571-a840-84072944f0d0 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:40.972370 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4940aa18-f7a9-4571-a840-84072944f0d0 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:40.972645 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4940aa18-f7a9-4571-a840-84072944f0d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:41.000102 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-4940aa18-f7a9-4571-a840-84072944f0d0 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:46:41.001688 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-4940aa18-f7a9-4571-a840-84072944f0d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:46:41.048190 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:41.053296 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b419120b-8333-43f0-87be-c213f90f8834 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:41.096678 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 156/318] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:46:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 11:46:41.099146 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4940aa18-f7a9-4571-a840-84072944f0d0 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:41.111166 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f351a43-5fd4-4740-83b5-6e48d6ce8f49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:41.111793 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f351a43-5fd4-4740-83b5-6e48d6ce8f49 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:41.112126 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f351a43-5fd4-4740-83b5-6e48d6ce8f49 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:41.112371 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f351a43-5fd4-4740-83b5-6e48d6ce8f49 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:41.113112 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f351a43-5fd4-4740-83b5-6e48d6ce8f49 None admin] RBAC: Adding query filter params (group.id=5e917abb7b794d318682da2487edff33, role.id=3a1a5476791547419a0b885c2022ef68, scope.project.id=1a4b3d97a66e4af4be0a066285f102de) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:46:41.113413 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f351a43-5fd4-4740-83b5-6e48d6ce8f49 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:41.114928 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f351a43-5fd4-4740-83b5-6e48d6ce8f49 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:41.123983 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 163/319] 10.222.0.59 () {56 vars in 1376 bytes} [Tue Oct 11 11:46:40 2022] GET /identity/v3/role_assignments?group.id=5e917abb7b794d318682da2487edff33&scope.project.id=1a4b3d97a66e4af4be0a066285f102de&role.id=3a1a5476791547419a0b885c2022ef68 => generated 611 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:46:41.136742 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f351a43-5fd4-4740-83b5-6e48d6ce8f49 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:41.203645 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:41.294467 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4940aa18-f7a9-4571-a840-84072944f0d0 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:41.309864 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f351a43-5fd4-4740-83b5-6e48d6ce8f49 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:41.384825 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608b0753-85d2-4e0e-b87f-627d723b4e66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:41.385780 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608b0753-85d2-4e0e-b87f-627d723b4e66 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:41.386260 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608b0753-85d2-4e0e-b87f-627d723b4e66 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:41.386790 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608b0753-85d2-4e0e-b87f-627d723b4e66 None admin] PATH_INFO: `/v3/roles` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:41.388015 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608b0753-85d2-4e0e-b87f-627d723b4e66 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:46:41.388556 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608b0753-85d2-4e0e-b87f-627d723b4e66 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:41.391589 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608b0753-85d2-4e0e-b87f-627d723b4e66 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:41.403711 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f7e40f-0a82-43e9-b84e-eb0b5ab155a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:41.405004 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f7e40f-0a82-43e9-b84e-eb0b5ab155a2 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:41.405529 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f7e40f-0a82-43e9-b84e-eb0b5ab155a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:41.406076 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f7e40f-0a82-43e9-b84e-eb0b5ab155a2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:41.408482 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 157/320] 10.222.0.59 () {56 vars in 1111 bytes} [Tue Oct 11 11:46:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:46:41.411216 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608b0753-85d2-4e0e-b87f-627d723b4e66 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:41.424916 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f7e40f-0a82-43e9-b84e-eb0b5ab155a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 11:46:41.431011 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application [None req-38f7e40f-0a82-43e9-b84e-eb0b5ab155a2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 11 11:46:41.431011 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:46:41.431011 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:46:41.431011 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:46:41.431011 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:46:41.431011 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:46:41.431011 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:46:41.431011 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:46:41.431011 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:46:41.431011 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:46:41.431011 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:46:41.431011 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:46:41.431011 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 11:46:41.431011 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 11:46:41.431011 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 11:46:41.431011 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 11:46:41.432124 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 11:46:41.432124 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 11:46:41.432124 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 11:46:41.432124 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 11:46:41.432124 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:46:41.432124 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:46:41.432124 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:46:41.432124 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:46:41.432124 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:46:41.432124 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:46:41.432124 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:46:41.432124 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application with Lock( Oct 11 11:46:41.432124 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:46:41.432124 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:46:41.432124 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:46:41.432124 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:46:41.433092 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:46:41.433092 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:46:41.433092 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:46:41.433092 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:46:41.433092 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 11:46:41.433092 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 11:46:41.433092 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 11:46:41.433092 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 11:46:41.433092 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 11:46:41.433092 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 11:46:41.433092 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 11 11:46:41.433092 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:46:41.435303 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 164/321] 10.222.0.59 () {56 vars in 1099 bytes} [Tue Oct 11 11:46:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:46:41.437152 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f7e40f-0a82-43e9-b84e-eb0b5ab155a2 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:41.469796 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:41.568566 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608b0753-85d2-4e0e-b87f-627d723b4e66 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:41.599505 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f7e40f-0a82-43e9-b84e-eb0b5ab155a2 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:41.610762 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:41.615255 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f31466-0339-44a5-b016-8776378c4c54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:41.617066 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f31466-0339-44a5-b016-8776378c4c54 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:41.617508 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f31466-0339-44a5-b016-8776378c4c54 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:41.617866 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f31466-0339-44a5-b016-8776378c4c54 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:41.632764 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f31466-0339-44a5-b016-8776378c4c54 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 11:46:41.639290 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application [None req-b9f31466-0339-44a5-b016-8776378c4c54 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 11 11:46:41.639290 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:46:41.639290 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:46:41.639290 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:46:41.639290 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:46:41.639290 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:46:41.639290 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:46:41.639290 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:46:41.639290 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:46:41.639290 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:46:41.639290 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:46:41.639290 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:46:41.639290 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 11:46:41.639290 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 11:46:41.639290 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 11:46:41.639290 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 11:46:41.640543 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 11:46:41.640543 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 11:46:41.640543 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 11:46:41.640543 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 11:46:41.640543 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:46:41.640543 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:46:41.640543 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:46:41.640543 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:46:41.640543 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:46:41.640543 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:46:41.640543 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:46:41.640543 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application with Lock( Oct 11 11:46:41.640543 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:46:41.640543 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:46:41.640543 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:46:41.640543 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:46:41.641767 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:46:41.641767 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:46:41.641767 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:46:41.641767 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:46:41.641767 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 11:46:41.641767 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 11:46:41.641767 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 11:46:41.641767 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 11:46:41.641767 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 11:46:41.641767 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 11:46:41.641767 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 11 11:46:41.641767 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:46:41.641767 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 158/322] 10.222.0.59 () {56 vars in 1096 bytes} [Tue Oct 11 11:46:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:46:41.641767 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f31466-0339-44a5-b016-8776378c4c54 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:41.654723 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d7b24b-fb28-4c48-ae55-3942de1e4a7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:41.655354 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d7b24b-fb28-4c48-ae55-3942de1e4a7c None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:41.655639 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d7b24b-fb28-4c48-ae55-3942de1e4a7c None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:41.655909 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d7b24b-fb28-4c48-ae55-3942de1e4a7c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:41.748266 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d7b24b-fb28-4c48-ae55-3942de1e4a7c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:41.749787 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d7b24b-fb28-4c48-ae55-3942de1e4a7c None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:41.778928 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:41.789956 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f31466-0339-44a5-b016-8776378c4c54 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:41.834182 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79be95eb-846d-409e-ac5d-ed1d81a6aa89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:41.835031 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79be95eb-846d-409e-ac5d-ed1d81a6aa89 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:41.835310 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79be95eb-846d-409e-ac5d-ed1d81a6aa89 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:41.835577 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79be95eb-846d-409e-ac5d-ed1d81a6aa89 None admin] PATH_INFO: `/v3/roles` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:41.836261 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79be95eb-846d-409e-ac5d-ed1d81a6aa89 None admin] RBAC: Adding query filter params (name=member) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:46:41.836560 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79be95eb-846d-409e-ac5d-ed1d81a6aa89 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:41.838717 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79be95eb-846d-409e-ac5d-ed1d81a6aa89 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:41.856726 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 159/323] 10.222.0.59 () {56 vars in 1101 bytes} [Tue Oct 11 11:46:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:46:41.859008 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79be95eb-846d-409e-ac5d-ed1d81a6aa89 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:41.881436 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 165/324] 10.222.0.59 () {58 vars in 1282 bytes} [Tue Oct 11 11:46:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 11 11:46:41.882183 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d7b24b-fb28-4c48-ae55-3942de1e4a7c None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:41.893599 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:42.004035 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79be95eb-846d-409e-ac5d-ed1d81a6aa89 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:42.042645 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d7b24b-fb28-4c48-ae55-3942de1e4a7c None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:42.060562 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc113c05-3c0e-42e4-b3f7-6223acb8e553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:42.061190 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc113c05-3c0e-42e4-b3f7-6223acb8e553 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:42.061466 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc113c05-3c0e-42e4-b3f7-6223acb8e553 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:42.061736 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc113c05-3c0e-42e4-b3f7-6223acb8e553 None admin] PATH_INFO: `/v3/roles` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:42.062410 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc113c05-3c0e-42e4-b3f7-6223acb8e553 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:46:42.062771 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc113c05-3c0e-42e4-b3f7-6223acb8e553 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:42.064163 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc113c05-3c0e-42e4-b3f7-6223acb8e553 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:42.073015 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 160/325] 10.222.0.59 () {56 vars in 1099 bytes} [Tue Oct 11 11:46:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:46:42.073872 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc113c05-3c0e-42e4-b3f7-6223acb8e553 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:42.086298 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1ecc49-8574-444c-b2e2-afafefa12fd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:42.086957 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1ecc49-8574-444c-b2e2-afafefa12fd0 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:42.087227 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1ecc49-8574-444c-b2e2-afafefa12fd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:42.087495 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1ecc49-8574-444c-b2e2-afafefa12fd0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:42.099597 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:42.168919 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1ecc49-8574-444c-b2e2-afafefa12fd0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:42.170335 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1ecc49-8574-444c-b2e2-afafefa12fd0 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:42.181940 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:42.185433 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc113c05-3c0e-42e4-b3f7-6223acb8e553 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:42.223455 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0bfb57-f25f-4cd7-9bcd-825fd654511d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:42.224110 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0bfb57-f25f-4cd7-9bcd-825fd654511d None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:42.224392 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0bfb57-f25f-4cd7-9bcd-825fd654511d None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:42.224669 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0bfb57-f25f-4cd7-9bcd-825fd654511d None admin] PATH_INFO: `/v3/projects/ed233e6a85f94375aece3faac28df210` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:42.225917 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0bfb57-f25f-4cd7-9bcd-825fd654511d None admin] RBAC: Authorizing `identity:get_project(project_id=ed233e6a85f94375aece3faac28df210)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:42.227439 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0bfb57-f25f-4cd7-9bcd-825fd654511d None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:42.229124 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 161/326] 10.222.0.59 () {56 vars in 1186 bytes} [Tue Oct 11 11:46:42 2022] GET /identity/v3/projects/ed233e6a85f94375aece3faac28df210 => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:46:42.229934 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0bfb57-f25f-4cd7-9bcd-825fd654511d None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:42.309408 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 166/327] 10.222.0.59 () {58 vars in 1282 bytes} [Tue Oct 11 11:46:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 11 11:46:42.310323 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1ecc49-8574-444c-b2e2-afafefa12fd0 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:42.321758 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:42.338049 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0bfb57-f25f-4cd7-9bcd-825fd654511d None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:42.374275 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5851e80-9e39-43d1-a72f-514496270d3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:42.374950 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5851e80-9e39-43d1-a72f-514496270d3c None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:42.375227 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5851e80-9e39-43d1-a72f-514496270d3c None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:42.375503 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5851e80-9e39-43d1-a72f-514496270d3c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:42.431378 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1ecc49-8574-444c-b2e2-afafefa12fd0 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:42.446991 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:42.463161 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5851e80-9e39-43d1-a72f-514496270d3c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:42.464676 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5851e80-9e39-43d1-a72f-514496270d3c None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:42.485996 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10eafdb4-c3d8-4138-8e06-125d86d4d120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:42.487030 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eafdb4-c3d8-4138-8e06-125d86d4d120 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:42.487477 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eafdb4-c3d8-4138-8e06-125d86d4d120 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:42.487899 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eafdb4-c3d8-4138-8e06-125d86d4d120 None admin] PATH_INFO: `/v3/groups/5e917abb7b794d318682da2487edff33` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:42.489737 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10eafdb4-c3d8-4138-8e06-125d86d4d120 None admin] RBAC: Authorizing `identity:get_group(group_id=5e917abb7b794d318682da2487edff33)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:42.492375 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10eafdb4-c3d8-4138-8e06-125d86d4d120 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:42.495291 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 167/328] 10.222.0.59 () {56 vars in 1180 bytes} [Tue Oct 11 11:46:42 2022] GET /identity/v3/groups/5e917abb7b794d318682da2487edff33 => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:46:42.496374 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10eafdb4-c3d8-4138-8e06-125d86d4d120 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:42.574135 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:42.594486 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 162/329] 10.222.0.59 () {58 vars in 1282 bytes} [Tue Oct 11 11:46:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 11 11:46:42.595461 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5851e80-9e39-43d1-a72f-514496270d3c None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:42.643935 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10eafdb4-c3d8-4138-8e06-125d86d4d120 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:42.681730 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa2d96f-7d18-48d0-9bb1-ee6f38e5ca06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:42.682688 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa2d96f-7d18-48d0-9bb1-ee6f38e5ca06 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:42.683141 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa2d96f-7d18-48d0-9bb1-ee6f38e5ca06 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:42.683410 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa2d96f-7d18-48d0-9bb1-ee6f38e5ca06 None admin] PATH_INFO: `/v3/projects/ed233e6a85f94375aece3faac28df210` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:42.684656 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa2d96f-7d18-48d0-9bb1-ee6f38e5ca06 None admin] RBAC: Authorizing `identity:get_project(project_id=ed233e6a85f94375aece3faac28df210)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:42.686138 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa2d96f-7d18-48d0-9bb1-ee6f38e5ca06 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:42.687902 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 168/330] 10.222.0.59 () {56 vars in 1186 bytes} [Tue Oct 11 11:46:42 2022] GET /identity/v3/projects/ed233e6a85f94375aece3faac28df210 => generated 302 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:46:42.688722 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa2d96f-7d18-48d0-9bb1-ee6f38e5ca06 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:42.718581 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5851e80-9e39-43d1-a72f-514496270d3c None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:42.750175 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:42.756679 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09623e36-f038-4140-b44c-630cdb450e8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:42.757301 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09623e36-f038-4140-b44c-630cdb450e8c None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:42.757582 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09623e36-f038-4140-b44c-630cdb450e8c None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:42.757847 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09623e36-f038-4140-b44c-630cdb450e8c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:42.758620 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09623e36-f038-4140-b44c-630cdb450e8c None admin] RBAC: Adding query filter params (group.id=5e917abb7b794d318682da2487edff33, role.id=3a1a5476791547419a0b885c2022ef68, scope.project.id=ed233e6a85f94375aece3faac28df210) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:46:42.758934 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09623e36-f038-4140-b44c-630cdb450e8c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:42.760425 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09623e36-f038-4140-b44c-630cdb450e8c None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:42.766878 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 163/331] 10.222.0.59 () {56 vars in 1376 bytes} [Tue Oct 11 11:46:42 2022] GET /identity/v3/role_assignments?group.id=5e917abb7b794d318682da2487edff33&scope.project.id=ed233e6a85f94375aece3faac28df210&role.id=3a1a5476791547419a0b885c2022ef68 => generated 611 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:46:42.767752 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09623e36-f038-4140-b44c-630cdb450e8c None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:42.838197 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:42.846754 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa2d96f-7d18-48d0-9bb1-ee6f38e5ca06 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:42.898319 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4065324a-d534-43d1-92dc-dae0eab0f0ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:42.899100 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4065324a-d534-43d1-92dc-dae0eab0f0ee None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:42.899378 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4065324a-d534-43d1-92dc-dae0eab0f0ee None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:42.899771 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4065324a-d534-43d1-92dc-dae0eab0f0ee None admin] PATH_INFO: `/v3/projects/ec9b5254352a410fb1f7cca5a5393294` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:42.901210 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4065324a-d534-43d1-92dc-dae0eab0f0ee None admin] RBAC: Authorizing `identity:get_project(project_id=ec9b5254352a410fb1f7cca5a5393294)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:42.903426 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4065324a-d534-43d1-92dc-dae0eab0f0ee None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:42.910947 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 169/332] 10.222.0.59 () {56 vars in 1186 bytes} [Tue Oct 11 11:46:42 2022] GET /identity/v3/projects/ec9b5254352a410fb1f7cca5a5393294 => generated 344 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:46:42.911969 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4065324a-d534-43d1-92dc-dae0eab0f0ee None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:42.916296 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09623e36-f038-4140-b44c-630cdb450e8c None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:42.960734 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fba93b-273b-4a10-8f0d-12bd43484c36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:42.961383 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fba93b-273b-4a10-8f0d-12bd43484c36 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:42.961658 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fba93b-273b-4a10-8f0d-12bd43484c36 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:42.962015 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fba93b-273b-4a10-8f0d-12bd43484c36 None admin] PATH_INFO: `/v3/groups/5e917abb7b794d318682da2487edff33` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:42.963370 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fba93b-273b-4a10-8f0d-12bd43484c36 None admin] RBAC: Authorizing `identity:get_group(group_id=5e917abb7b794d318682da2487edff33)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:42.965693 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fba93b-273b-4a10-8f0d-12bd43484c36 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:42.967456 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 164/333] 10.222.0.59 () {56 vars in 1180 bytes} [Tue Oct 11 11:46:42 2022] GET /identity/v3/groups/5e917abb7b794d318682da2487edff33 => generated 226 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:46:42.974156 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fba93b-273b-4a10-8f0d-12bd43484c36 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:43.060271 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4065324a-d534-43d1-92dc-dae0eab0f0ee None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:43.082702 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:43.113920 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fba93b-273b-4a10-8f0d-12bd43484c36 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:43.114235 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a984e33-a791-4666-a381-6c5e3f68a1b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:43.114966 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a984e33-a791-4666-a381-6c5e3f68a1b5 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:43.115221 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a984e33-a791-4666-a381-6c5e3f68a1b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:43.115509 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a984e33-a791-4666-a381-6c5e3f68a1b5 None admin] PATH_INFO: `/v3/groups/61c402f2dab14543932c242c96eb8782` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:43.116752 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a984e33-a791-4666-a381-6c5e3f68a1b5 None admin] RBAC: Authorizing `identity:get_group(group_id=61c402f2dab14543932c242c96eb8782)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:43.118302 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a984e33-a791-4666-a381-6c5e3f68a1b5 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:43.120154 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 170/334] 10.222.0.59 () {56 vars in 1180 bytes} [Tue Oct 11 11:46:42 2022] GET /identity/v3/groups/61c402f2dab14543932c242c96eb8782 => generated 229 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:46:43.126475 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a984e33-a791-4666-a381-6c5e3f68a1b5 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:43.136077 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:43.156765 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996d0693-c838-4f39-9a3d-32c6fa2baae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:43.157413 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996d0693-c838-4f39-9a3d-32c6fa2baae8 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:43.157682 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996d0693-c838-4f39-9a3d-32c6fa2baae8 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:43.157956 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996d0693-c838-4f39-9a3d-32c6fa2baae8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:43.158757 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996d0693-c838-4f39-9a3d-32c6fa2baae8 None admin] RBAC: Adding query filter params (group.id=5e917abb7b794d318682da2487edff33, role.id=de2d2bd26fff4748a79990f97a107164, scope.project.id=ed233e6a85f94375aece3faac28df210) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:46:43.159055 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996d0693-c838-4f39-9a3d-32c6fa2baae8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:43.160989 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996d0693-c838-4f39-9a3d-32c6fa2baae8 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:43.167608 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 165/335] 10.222.0.59 () {56 vars in 1376 bytes} [Tue Oct 11 11:46:42 2022] GET /identity/v3/role_assignments?group.id=5e917abb7b794d318682da2487edff33&scope.project.id=ed233e6a85f94375aece3faac28df210&role.id=de2d2bd26fff4748a79990f97a107164 => generated 611 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:46:43.250928 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a984e33-a791-4666-a381-6c5e3f68a1b5 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:43.297421 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c8ebc5-4005-4e50-be29-a447cc6a199c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:43.298056 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c8ebc5-4005-4e50-be29-a447cc6a199c None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:43.298309 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c8ebc5-4005-4e50-be29-a447cc6a199c None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:43.298640 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c8ebc5-4005-4e50-be29-a447cc6a199c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:43.299391 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c8ebc5-4005-4e50-be29-a447cc6a199c None admin] RBAC: Adding query filter params (group.id=61c402f2dab14543932c242c96eb8782, role.id=5cf1f4bd2fef46ab9c6eff56d52f9f69, scope.project.id=ec9b5254352a410fb1f7cca5a5393294) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:46:43.299702 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c8ebc5-4005-4e50-be29-a447cc6a199c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:43.301230 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c8ebc5-4005-4e50-be29-a447cc6a199c None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:43.307992 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 171/336] 10.222.0.59 () {56 vars in 1376 bytes} [Tue Oct 11 11:46:43 2022] GET /identity/v3/role_assignments?group.id=61c402f2dab14543932c242c96eb8782&scope.project.id=ec9b5254352a410fb1f7cca5a5393294&role.id=5cf1f4bd2fef46ab9c6eff56d52f9f69 => generated 611 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 11:46:46.516752 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3978538-3a30-47da-aede-167e92152471 None None] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:46.517314 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3978538-3a30-47da-aede-167e92152471 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:46.517314 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3978538-3a30-47da-aede-167e92152471 None None] PATH_INFO: `/` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:46.519057 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 166/337] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:46:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:46:46.528393 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4275482f-d7aa-4e75-bf25-8499ad03b572 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:46.529898 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4275482f-d7aa-4e75-bf25-8499ad03b572 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:46.530683 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4275482f-d7aa-4e75-bf25-8499ad03b572 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:46.564103 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-4275482f-d7aa-4e75-bf25-8499ad03b572 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:46:46.566651 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-4275482f-d7aa-4e75-bf25-8499ad03b572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:46:46.639831 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:46.694718 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 172/338] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:46:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 11:46:47.143205 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69f5e2e-3318-4fd2-a392-28b85e55e7eb None None] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:47.143943 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69f5e2e-3318-4fd2-a392-28b85e55e7eb None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:47.144435 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69f5e2e-3318-4fd2-a392-28b85e55e7eb None None] PATH_INFO: `/` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:47.146455 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 167/339] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:46:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:46:47.152979 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0621f6b-c59b-46a9-8ffa-17fa2acc5e93 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:47.153634 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0621f6b-c59b-46a9-8ffa-17fa2acc5e93 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:47.154172 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0621f6b-c59b-46a9-8ffa-17fa2acc5e93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:47.179632 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-e0621f6b-c59b-46a9-8ffa-17fa2acc5e93 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:46:47.180859 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-e0621f6b-c59b-46a9-8ffa-17fa2acc5e93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:46:47.223855 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69f5e2e-3318-4fd2-a392-28b85e55e7eb None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:47.336006 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 173/340] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:46:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 11:46:47.349126 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b016f7e0-c94f-4ddd-96cd-1b7132e952e0 None None] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:47.349576 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b016f7e0-c94f-4ddd-96cd-1b7132e952e0 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:47.350026 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b016f7e0-c94f-4ddd-96cd-1b7132e952e0 None None] PATH_INFO: `/` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:47.352959 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 174/341] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:46:47 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:46:47.363625 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c637d42e-0e66-4f6c-9012-f3e6c0a6d38d None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:47.364622 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c637d42e-0e66-4f6c-9012-f3e6c0a6d38d None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:47.365285 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c637d42e-0e66-4f6c-9012-f3e6c0a6d38d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:47.419771 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-c637d42e-0e66-4f6c-9012-f3e6c0a6d38d None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:46:47.421018 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-c637d42e-0e66-4f6c-9012-f3e6c0a6d38d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:46:47.488199 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69f5e2e-3318-4fd2-a392-28b85e55e7eb None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:47.559098 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a0f23f-1143-49ad-932c-a0167a8fcaf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:47.561342 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a0f23f-1143-49ad-932c-a0167a8fcaf7 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:47.562073 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a0f23f-1143-49ad-932c-a0167a8fcaf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:47.562788 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a0f23f-1143-49ad-932c-a0167a8fcaf7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:47.587579 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a0f23f-1143-49ad-932c-a0167a8fcaf7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:47.590855 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a0f23f-1143-49ad-932c-a0167a8fcaf7 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:47.626887 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application [None req-22a0f23f-1143-49ad-932c-a0167a8fcaf7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 11:46:47.626887 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:46:47.626887 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:46:47.626887 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:46:47.626887 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:46:47.626887 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:46:47.626887 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:46:47.626887 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:46:47.626887 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:46:47.626887 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:46:47.626887 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:46:47.626887 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:46:47.626887 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 11:46:47.626887 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 11:46:47.626887 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 11:46:47.626887 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 11:46:47.629072 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:46:47.629072 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:46:47.629072 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:46:47.629072 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:46:47.629072 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:46:47.629072 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:46:47.629072 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:46:47.629072 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application with Lock( Oct 11 11:46:47.629072 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:46:47.629072 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:46:47.629072 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:46:47.629072 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:46:47.629072 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:46:47.629072 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:46:47.629072 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:46:47.629072 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:46:47.629072 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 11:46:47.631077 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 11:46:47.631077 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 11:46:47.631077 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:46:47.634342 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 168/342] 10.222.0.59 () {56 vars in 1108 bytes} [Tue Oct 11 11:46:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 411 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 11 11:46:47.644518 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a0f23f-1143-49ad-932c-a0167a8fcaf7 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:47.649483 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 175/343] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:46:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 11 11:46:47.735408 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:47.818247 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c637d42e-0e66-4f6c-9012-f3e6c0a6d38d None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:47.873486 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a0f23f-1143-49ad-932c-a0167a8fcaf7 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:47.874419 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:47.943288 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4a4154-081f-45c1-98ad-10f3dc8e5d04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:47.944814 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4a4154-081f-45c1-98ad-10f3dc8e5d04 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:47.945534 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4a4154-081f-45c1-98ad-10f3dc8e5d04 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:47.946187 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4a4154-081f-45c1-98ad-10f3dc8e5d04 None admin] PATH_INFO: `/v3/domains` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:47.947770 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4a4154-081f-45c1-98ad-10f3dc8e5d04 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:46:47.948538 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4a4154-081f-45c1-98ad-10f3dc8e5d04 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:47.951875 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4a4154-081f-45c1-98ad-10f3dc8e5d04 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:47.976545 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 169/344] 10.222.0.59 () {56 vars in 1109 bytes} [Tue Oct 11 11:46:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:46:47.979416 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41696425-cc39-4259-9884-8970e03b719d None None] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:47.980203 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41696425-cc39-4259-9884-8970e03b719d None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:47.980919 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41696425-cc39-4259-9884-8970e03b719d None None] PATH_INFO: `/` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:47.989765 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 170/345] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:46:47 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:46:47.989765 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41696425-cc39-4259-9884-8970e03b719d None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:48.024531 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c637d42e-0e66-4f6c-9012-f3e6c0a6d38d None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:48.092944 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2828b117-2cfe-4a2f-8be4-8e6423bafc2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:48.093945 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2828b117-2cfe-4a2f-8be4-8e6423bafc2a None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:48.094435 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2828b117-2cfe-4a2f-8be4-8e6423bafc2a None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:48.094863 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2828b117-2cfe-4a2f-8be4-8e6423bafc2a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:48.113238 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2828b117-2cfe-4a2f-8be4-8e6423bafc2a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:48.114454 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:48.115961 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2828b117-2cfe-4a2f-8be4-8e6423bafc2a None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:48.124886 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41696425-cc39-4259-9884-8970e03b719d None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:48.143974 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application [None req-2828b117-2cfe-4a2f-8be4-8e6423bafc2a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 11:46:48.143974 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:46:48.143974 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:46:48.143974 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:46:48.143974 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:46:48.143974 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:46:48.143974 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:46:48.143974 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:46:48.143974 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:46:48.143974 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:46:48.143974 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:46:48.143974 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:46:48.143974 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 11:46:48.143974 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 11:46:48.143974 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 11:46:48.143974 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 11:46:48.145683 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:46:48.145683 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:46:48.145683 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:46:48.145683 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:46:48.145683 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:46:48.145683 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:46:48.145683 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:46:48.145683 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application with Lock( Oct 11 11:46:48.145683 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:46:48.145683 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:46:48.145683 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:46:48.145683 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:46:48.145683 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:46:48.145683 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:46:48.145683 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:46:48.145683 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:46:48.145683 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 11:46:48.147422 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 11:46:48.147422 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 11:46:48.147422 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:46:48.147422 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 176/346] 10.222.0.59 () {56 vars in 1108 bytes} [Tue Oct 11 11:46:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:46:48.147836 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2828b117-2cfe-4a2f-8be4-8e6423bafc2a None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:48.176652 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178c53af-403e-4d32-9a7e-1102d4dc601a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:48.177763 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178c53af-403e-4d32-9a7e-1102d4dc601a None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:48.178257 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178c53af-403e-4d32-9a7e-1102d4dc601a None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:48.178878 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178c53af-403e-4d32-9a7e-1102d4dc601a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:48.198140 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178c53af-403e-4d32-9a7e-1102d4dc601a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:48.201860 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178c53af-403e-4d32-9a7e-1102d4dc601a None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:48.233142 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application [None req-178c53af-403e-4d32-9a7e-1102d4dc601a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 11:46:48.233142 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:46:48.233142 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:46:48.233142 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:46:48.233142 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:46:48.233142 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:46:48.233142 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:46:48.233142 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:46:48.233142 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:46:48.233142 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:46:48.233142 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:46:48.233142 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:46:48.233142 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 11:46:48.233142 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 11:46:48.233142 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 11:46:48.233142 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 11:46:48.235311 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:46:48.235311 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:46:48.235311 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:46:48.235311 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:46:48.235311 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:46:48.235311 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:46:48.235311 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:46:48.235311 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application with Lock( Oct 11 11:46:48.235311 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:46:48.235311 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:46:48.235311 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:46:48.235311 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:46:48.235311 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:46:48.235311 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:46:48.235311 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:46:48.235311 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:46:48.235311 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 11:46:48.237111 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 11:46:48.237111 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 11:46:48.237111 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:46:48.242695 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 171/347] 10.222.0.59 () {56 vars in 1108 bytes} [Tue Oct 11 11:46:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:46:48.244652 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94093b10-c78b-4b62-a59a-bd22c8fffd53 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:48.245065 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94093b10-c78b-4b62-a59a-bd22c8fffd53 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:48.245457 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94093b10-c78b-4b62-a59a-bd22c8fffd53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:48.276308 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-94093b10-c78b-4b62-a59a-bd22c8fffd53 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:46:48.278416 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-94093b10-c78b-4b62-a59a-bd22c8fffd53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:46:48.296402 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:48.321646 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2828b117-2cfe-4a2f-8be4-8e6423bafc2a None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:48.376829 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823efe75-ff33-484d-89c8-ae5fb66d6462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:48.377825 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823efe75-ff33-484d-89c8-ae5fb66d6462 None admin] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:48.378303 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823efe75-ff33-484d-89c8-ae5fb66d6462 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:48.378825 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823efe75-ff33-484d-89c8-ae5fb66d6462 None admin] PATH_INFO: `/v3/users` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:48.379991 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823efe75-ff33-484d-89c8-ae5fb66d6462 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:48.382666 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823efe75-ff33-484d-89c8-ae5fb66d6462 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:48.395453 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 172/348] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:46:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 11:46:48.396830 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94093b10-c78b-4b62-a59a-bd22c8fffd53 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:48.426796 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 177/349] 10.222.0.59 () {60 vars in 1132 bytes} [Tue Oct 11 11:46:48 2022] POST /identity/v3/users => generated 248 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 11:46:48.428176 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823efe75-ff33-484d-89c8-ae5fb66d6462 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:48.586368 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:48.642337 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94093b10-c78b-4b62-a59a-bd22c8fffd53 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:48.690011 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823efe75-ff33-484d-89c8-ae5fb66d6462 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:48.723313 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10d161f-5136-4fa9-ae57-55fa9131ed90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:48.725372 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10d161f-5136-4fa9-ae57-55fa9131ed90 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:48.726317 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10d161f-5136-4fa9-ae57-55fa9131ed90 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:48.727255 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10d161f-5136-4fa9-ae57-55fa9131ed90 None admin] PATH_INFO: `/v3/domains` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:48.728769 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10d161f-5136-4fa9-ae57-55fa9131ed90 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:46:48.729629 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10d161f-5136-4fa9-ae57-55fa9131ed90 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:48.732823 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10d161f-5136-4fa9-ae57-55fa9131ed90 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:48.760835 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 173/350] 10.222.0.59 () {56 vars in 1109 bytes} [Tue Oct 11 11:46:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 365 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:46:48.765815 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cec97c5-af7f-4281-a213-b7eaae8b1d20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:48.769132 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10d161f-5136-4fa9-ae57-55fa9131ed90 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:48.770007 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cec97c5-af7f-4281-a213-b7eaae8b1d20 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:48.770525 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cec97c5-af7f-4281-a213-b7eaae8b1d20 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:48.771029 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cec97c5-af7f-4281-a213-b7eaae8b1d20 None admin] PATH_INFO: `/v3/domains` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:48.772226 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cec97c5-af7f-4281-a213-b7eaae8b1d20 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:46:48.772779 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cec97c5-af7f-4281-a213-b7eaae8b1d20 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:48.775419 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cec97c5-af7f-4281-a213-b7eaae8b1d20 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:48.826395 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 178/351] 10.222.0.59 () {56 vars in 1109 bytes} [Tue Oct 11 11:46:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 392 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:46:48.826395 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb0b656-0d6f-42fe-8d3c-a2ad4e77ccdf None None] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:48.826395 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb0b656-0d6f-42fe-8d3c-a2ad4e77ccdf None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:48.826395 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb0b656-0d6f-42fe-8d3c-a2ad4e77ccdf None None] PATH_INFO: `/` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:48.826395 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 179/352] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:46:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:46:48.828779 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb0b656-0d6f-42fe-8d3c-a2ad4e77ccdf None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:48.831431 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:48.949783 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10d161f-5136-4fa9-ae57-55fa9131ed90 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:48.988021 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb0b656-0d6f-42fe-8d3c-a2ad4e77ccdf None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:49.012276 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5678ef0-4d1c-4599-9eb1-7255147f69c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:49.013046 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5678ef0-4d1c-4599-9eb1-7255147f69c9 None admin] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:49.013504 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5678ef0-4d1c-4599-9eb1-7255147f69c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:49.013944 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5678ef0-4d1c-4599-9eb1-7255147f69c9 None admin] PATH_INFO: `/v3/users` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:49.015028 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5678ef0-4d1c-4599-9eb1-7255147f69c9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:49.017453 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5678ef0-4d1c-4599-9eb1-7255147f69c9 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:49.031164 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:49.047880 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2f6622-b9ff-4601-8e51-ea84d1c0e9a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:49.048601 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 174/353] 10.222.0.59 () {60 vars in 1132 bytes} [Tue Oct 11 11:46:48 2022] POST /identity/v3/users => generated 246 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 11:46:49.048908 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2f6622-b9ff-4601-8e51-ea84d1c0e9a2 None admin] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:49.049441 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2f6622-b9ff-4601-8e51-ea84d1c0e9a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:49.049883 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2f6622-b9ff-4601-8e51-ea84d1c0e9a2 None admin] PATH_INFO: `/v3/users` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:49.050058 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d06caad-0c9a-4618-8eba-06029385a685 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:49.050302 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d06caad-0c9a-4618-8eba-06029385a685 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:49.050626 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d06caad-0c9a-4618-8eba-06029385a685 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:49.051052 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2f6622-b9ff-4601-8e51-ea84d1c0e9a2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:49.052853 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2f6622-b9ff-4601-8e51-ea84d1c0e9a2 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:49.087573 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-2d06caad-0c9a-4618-8eba-06029385a685 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:46:49.089038 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-2d06caad-0c9a-4618-8eba-06029385a685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:46:49.095186 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 180/354] 10.222.0.59 () {60 vars in 1132 bytes} [Tue Oct 11 11:46:48 2022] POST /identity/v3/users => generated 249 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 11:46:49.096861 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2f6622-b9ff-4601-8e51-ea84d1c0e9a2 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:49.238994 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 175/355] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:46:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 11:46:49.247561 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:49.330841 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2f6622-b9ff-4601-8e51-ea84d1c0e9a2 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:49.384089 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a6ef46-6a4b-45fb-8eec-1c889212dfbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:49.384761 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a6ef46-6a4b-45fb-8eec-1c889212dfbe None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:49.385036 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a6ef46-6a4b-45fb-8eec-1c889212dfbe None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:49.385308 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a6ef46-6a4b-45fb-8eec-1c889212dfbe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:49.398357 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a6ef46-6a4b-45fb-8eec-1c889212dfbe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:49.399951 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a6ef46-6a4b-45fb-8eec-1c889212dfbe None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:49.416260 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application [None req-62a6ef46-6a4b-45fb-8eec-1c889212dfbe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 11:46:49.416260 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:46:49.416260 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:46:49.416260 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:46:49.416260 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:46:49.416260 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:46:49.416260 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:46:49.416260 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:46:49.416260 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:46:49.416260 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:46:49.416260 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:46:49.416260 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:46:49.416260 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 11:46:49.416260 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 11:46:49.416260 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 11:46:49.416260 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 11:46:49.417241 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:46:49.417241 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:46:49.417241 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:46:49.417241 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:46:49.417241 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:46:49.417241 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:46:49.417241 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:46:49.417241 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application with Lock( Oct 11 11:46:49.417241 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:46:49.417241 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:46:49.417241 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:46:49.417241 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:46:49.417241 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:46:49.417241 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:46:49.417241 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:46:49.417241 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:46:49.417241 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 11:46:49.418324 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 11:46:49.418324 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 11:46:49.418324 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:46:49.418324 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 181/356] 10.222.0.59 () {56 vars in 1108 bytes} [Tue Oct 11 11:46:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:46:49.425023 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d06caad-0c9a-4618-8eba-06029385a685 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:49.550913 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d06caad-0c9a-4618-8eba-06029385a685 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:49.580407 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:49.603807 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3f2695-c435-4af4-8d68-352371c9e5e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:49.604801 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3f2695-c435-4af4-8d68-352371c9e5e2 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:49.604801 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3f2695-c435-4af4-8d68-352371c9e5e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:49.605082 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3f2695-c435-4af4-8d68-352371c9e5e2 None admin] PATH_INFO: `/v3/domains` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:49.605620 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3f2695-c435-4af4-8d68-352371c9e5e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:46:49.605924 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3f2695-c435-4af4-8d68-352371c9e5e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:49.607375 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3f2695-c435-4af4-8d68-352371c9e5e2 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:49.628386 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 176/357] 10.222.0.59 () {56 vars in 1109 bytes} [Tue Oct 11 11:46:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:46:49.635015 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a6ef46-6a4b-45fb-8eec-1c889212dfbe None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:49.663104 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3f2695-c435-4af4-8d68-352371c9e5e2 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:49.861559 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3f2695-c435-4af4-8d68-352371c9e5e2 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:49.907115 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a6ef46-6a4b-45fb-8eec-1c889212dfbe None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:49.922000 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:49.951764 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01ff804-a7c3-4830-9540-87a6e989aa7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:49.952778 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01ff804-a7c3-4830-9540-87a6e989aa7c None admin] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:49.953317 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01ff804-a7c3-4830-9540-87a6e989aa7c None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:49.953785 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01ff804-a7c3-4830-9540-87a6e989aa7c None admin] PATH_INFO: `/v3/users` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:49.954729 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01ff804-a7c3-4830-9540-87a6e989aa7c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:49.956786 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01ff804-a7c3-4830-9540-87a6e989aa7c None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:49.972541 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b221bf47-2023-406f-9891-5d31c7d81cb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:49.973405 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b221bf47-2023-406f-9891-5d31c7d81cb4 None admin] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:49.973642 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b221bf47-2023-406f-9891-5d31c7d81cb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:49.973901 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b221bf47-2023-406f-9891-5d31c7d81cb4 None admin] PATH_INFO: `/v3/roles` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:49.980144 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b221bf47-2023-406f-9891-5d31c7d81cb4 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:49.980144 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b221bf47-2023-406f-9891-5d31c7d81cb4 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:49.991435 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.sql.core [None req-b221bf47-2023-406f-9891-5d31c7d81cb4 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 11 11:46:49.991435 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 11 11:46:49.991435 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [parameters: {'id': '57d28a53156f45f8a7ad0f31f727ae6c', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 11 11:46:49.991435 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81433) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 11 11:46:50.001039 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 182/358] 10.222.0.59 () {60 vars in 1132 bytes} [Tue Oct 11 11:46:49 2022] POST /identity/v3/users => generated 248 bytes in 367 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 11:46:50.006439 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application [None req-b221bf47-2023-406f-9891-5d31c7d81cb4 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 11 11:46:50.006439 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:46:50.006439 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 11 11:46:50.006439 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 11 11:46:50.006439 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 11 11:46:50.006439 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 11 11:46:50.006439 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 11 11:46:50.006439 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application result = self._query(query) Oct 11 11:46:50.006439 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 11 11:46:50.006439 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application conn.query(q) Oct 11 11:46:50.006439 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 11 11:46:50.006439 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 11 11:46:50.006439 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 11 11:46:50.006439 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application result.read() Oct 11 11:46:50.006439 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 11 11:46:50.007474 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 11 11:46:50.007474 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 11 11:46:50.007474 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 11 11:46:50.007474 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 11 11:46:50.007474 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 11 11:46:50.007474 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 11 11:46:50.007474 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 11 11:46:50.007474 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 11 11:46:50.007474 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:46:50.007474 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 11 11:46:50.007474 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:46:50.007474 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:46:50.007474 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 11 11:46:50.007474 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 11 11:46:50.007474 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Oct 11 11:46:50.007474 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return ref.to_dict() Oct 11 11:46:50.007474 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 11 11:46:50.007474 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application next(self.gen) Oct 11 11:46:50.008407 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 11 11:46:50.008407 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application yield resource Oct 11 11:46:50.008407 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 11 11:46:50.008407 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application next(self.gen) Oct 11 11:46:50.008407 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 11 11:46:50.008407 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application self.session.rollback() Oct 11 11:46:50.008407 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 11 11:46:50.008407 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application self.force_reraise() Oct 11 11:46:50.008407 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 11 11:46:50.008407 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application raise self.value Oct 11 11:46:50.008407 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 11 11:46:50.008407 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 11 11:46:50.008407 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 11 11:46:50.008407 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application session.commit() Oct 11 11:46:50.008407 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 11 11:46:50.008407 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 11 11:46:50.008407 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 11 11:46:50.009450 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application self._prepare_impl() Oct 11 11:46:50.009450 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 11 11:46:50.009450 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application self.session.flush() Oct 11 11:46:50.009450 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 11 11:46:50.009450 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application self._flush(objects) Oct 11 11:46:50.009450 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 11 11:46:50.009450 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 11 11:46:50.009450 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 11 11:46:50.009450 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application compat.raise_( Oct 11 11:46:50.009450 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 11 11:46:50.009450 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application raise exception Oct 11 11:46:50.009450 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 11 11:46:50.009450 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application flush_context.execute() Oct 11 11:46:50.009450 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 11 11:46:50.009450 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application rec.execute(self) Oct 11 11:46:50.009450 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 11 11:46:50.009450 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 11 11:46:50.010483 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 11 11:46:50.010483 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 11 11:46:50.010483 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 11 11:46:50.010483 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 11 11:46:50.010483 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 11 11:46:50.010483 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 11 11:46:50.010483 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 11 11:46:50.010483 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 11 11:46:50.010483 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 11 11:46:50.010483 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 11 11:46:50.010483 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 11 11:46:50.010483 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 11 11:46:50.010483 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 11 11:46:50.010483 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 11 11:46:50.010483 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 11 11:46:50.010483 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application raise exception Oct 11 11:46:50.011468 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 11 11:46:50.011468 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 11 11:46:50.011468 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 11 11:46:50.011468 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 11 11:46:50.011468 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 11 11:46:50.011468 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application result = self._query(query) Oct 11 11:46:50.011468 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 11 11:46:50.011468 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application conn.query(q) Oct 11 11:46:50.011468 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 11 11:46:50.011468 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 11 11:46:50.011468 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 11 11:46:50.011468 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application result.read() Oct 11 11:46:50.011468 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 11 11:46:50.011468 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 11 11:46:50.011468 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 11 11:46:50.011468 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 11 11:46:50.012394 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 11 11:46:50.012394 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 11 11:46:50.012394 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 11 11:46:50.012394 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 11 11:46:50.012394 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 11 11:46:50.012394 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 11 11:46:50.012394 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application [parameters: {'id': '57d28a53156f45f8a7ad0f31f727ae6c', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 11 11:46:50.012394 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 11 11:46:50.012394 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:46:50.012394 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 11 11:46:50.012394 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:46:50.012394 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:46:50.012394 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:46:50.012394 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:46:50.012394 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:46:50.012394 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:46:50.013214 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:46:50.013214 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:46:50.013214 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:46:50.013214 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:46:50.013214 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:46:50.013214 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:46:50.013214 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Oct 11 11:46:50.013214 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Oct 11 11:46:50.013214 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:46:50.013214 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:46:50.013214 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Oct 11 11:46:50.013214 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Oct 11 11:46:50.013214 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 11 11:46:50.013214 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 11 11:46:50.013214 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 11 11:46:50.013214 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:46:50.016052 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 177/359] 10.222.0.59 () {60 vars in 1131 bytes} [Tue Oct 11 11:46:49 2022] POST /identity/v3/roles => generated 144 bytes in 354 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Oct 11 11:46:50.024997 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01ff804-a7c3-4830-9540-87a6e989aa7c None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:50.193599 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01ff804-a7c3-4830-9540-87a6e989aa7c None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:50.253205 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7179d1aa-ec2a-4afa-9172-f271ac6af800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:50.254171 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7179d1aa-ec2a-4afa-9172-f271ac6af800 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:50.254635 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7179d1aa-ec2a-4afa-9172-f271ac6af800 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:50.255053 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7179d1aa-ec2a-4afa-9172-f271ac6af800 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:50.273106 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7179d1aa-ec2a-4afa-9172-f271ac6af800 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 11:46:50.278635 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application [None req-7179d1aa-ec2a-4afa-9172-f271ac6af800 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 11 11:46:50.278635 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:46:50.278635 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:46:50.278635 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:46:50.278635 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:46:50.278635 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:46:50.278635 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:46:50.278635 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:46:50.278635 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:46:50.278635 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:46:50.278635 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:46:50.278635 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:46:50.278635 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 11:46:50.278635 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 11:46:50.278635 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 11:46:50.278635 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 11:46:50.279821 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 11:46:50.279821 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 11:46:50.279821 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 11:46:50.279821 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 11:46:50.279821 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:46:50.279821 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:46:50.279821 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:46:50.279821 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:46:50.279821 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:46:50.279821 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:46:50.279821 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:46:50.279821 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application with Lock( Oct 11 11:46:50.279821 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:46:50.279821 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:46:50.279821 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:46:50.279821 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:46:50.280830 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:46:50.280830 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:46:50.280830 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:46:50.280830 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:46:50.280830 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 11:46:50.280830 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 11:46:50.280830 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 11:46:50.280830 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 11:46:50.280830 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 11:46:50.280830 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 11:46:50.280830 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 11 11:46:50.280830 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:46:50.283040 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 183/360] 10.222.0.59 () {56 vars in 1114 bytes} [Tue Oct 11 11:46:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:46:50.290104 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b221bf47-2023-406f-9891-5d31c7d81cb4 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:50.344863 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:50.421971 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b221bf47-2023-406f-9891-5d31c7d81cb4 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:50.475440 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df8084b-97b0-4b9b-96bb-c1adaedfb55f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:50.475976 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df8084b-97b0-4b9b-96bb-c1adaedfb55f None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:50.476260 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df8084b-97b0-4b9b-96bb-c1adaedfb55f None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:50.476535 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df8084b-97b0-4b9b-96bb-c1adaedfb55f None admin] PATH_INFO: `/v3/roles` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:50.477227 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df8084b-97b0-4b9b-96bb-c1adaedfb55f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:46:50.477534 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df8084b-97b0-4b9b-96bb-c1adaedfb55f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:50.478957 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df8084b-97b0-4b9b-96bb-c1adaedfb55f None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:50.492952 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 178/361] 10.222.0.59 () {56 vars in 1111 bytes} [Tue Oct 11 11:46:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:46:51.327850 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7495504c-b389-46ff-98f7-8aa5859e2ebf None None] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:51.328604 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7495504c-b389-46ff-98f7-8aa5859e2ebf None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:51.328604 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7495504c-b389-46ff-98f7-8aa5859e2ebf None None] PATH_INFO: `/` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:51.330042 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 184/362] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:46:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:46:51.339365 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520fa81e-665e-4483-9243-7cff6e3c5245 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:51.340112 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520fa81e-665e-4483-9243-7cff6e3c5245 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:51.340112 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520fa81e-665e-4483-9243-7cff6e3c5245 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:51.379645 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-520fa81e-665e-4483-9243-7cff6e3c5245 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:46:51.380862 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-520fa81e-665e-4483-9243-7cff6e3c5245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:46:51.539646 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 179/363] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:46:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 11 11:46:51.906697 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7495504c-b389-46ff-98f7-8aa5859e2ebf None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:51.945582 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:52.075345 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7495504c-b389-46ff-98f7-8aa5859e2ebf None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:52.120414 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74840101-235a-4cf7-9f26-da0ebe8cea3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:52.121071 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74840101-235a-4cf7-9f26-da0ebe8cea3a None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:52.121347 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74840101-235a-4cf7-9f26-da0ebe8cea3a None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:52.121617 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74840101-235a-4cf7-9f26-da0ebe8cea3a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:52.145241 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74840101-235a-4cf7-9f26-da0ebe8cea3a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 11:46:52.150252 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application [None req-74840101-235a-4cf7-9f26-da0ebe8cea3a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 11 11:46:52.150252 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:46:52.150252 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:46:52.150252 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:46:52.150252 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:46:52.150252 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:46:52.150252 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:46:52.150252 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:46:52.150252 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:46:52.150252 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:46:52.150252 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:46:52.150252 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:46:52.150252 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 11:46:52.150252 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 11:46:52.150252 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 11:46:52.150252 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 11:46:52.152009 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 11:46:52.152009 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 11:46:52.152009 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 11:46:52.152009 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 11:46:52.152009 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:46:52.152009 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:46:52.152009 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:46:52.152009 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:46:52.152009 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:46:52.152009 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:46:52.152009 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:46:52.152009 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application with Lock( Oct 11 11:46:52.152009 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:46:52.152009 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:46:52.152009 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:46:52.152009 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:46:52.153586 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:46:52.153586 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:46:52.153586 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:46:52.153586 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:46:52.153586 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 11:46:52.153586 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 11:46:52.153586 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 11:46:52.153586 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 11:46:52.153586 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 11:46:52.153586 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 11:46:52.153586 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 11 11:46:52.153586 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:46:52.153586 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 185/364] 10.222.0.59 () {56 vars in 1102 bytes} [Tue Oct 11 11:46:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:46:52.158655 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520fa81e-665e-4483-9243-7cff6e3c5245 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:52.283133 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:52.380087 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520fa81e-665e-4483-9243-7cff6e3c5245 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:52.434703 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6a47e0-920a-4881-931e-5c3d2b4cc5bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:52.435232 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6a47e0-920a-4881-931e-5c3d2b4cc5bf None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:52.435511 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6a47e0-920a-4881-931e-5c3d2b4cc5bf None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:52.435781 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6a47e0-920a-4881-931e-5c3d2b4cc5bf None admin] PATH_INFO: `/v3/roles` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:52.436473 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6a47e0-920a-4881-931e-5c3d2b4cc5bf None admin] RBAC: Adding query filter params (name=service) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:46:52.436776 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6a47e0-920a-4881-931e-5c3d2b4cc5bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:52.438169 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6a47e0-920a-4881-931e-5c3d2b4cc5bf None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:52.450983 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 180/365] 10.222.0.59 () {56 vars in 1103 bytes} [Tue Oct 11 11:46:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 11:46:52.458010 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74840101-235a-4cf7-9f26-da0ebe8cea3a None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:52.617630 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97814e9f-ca2a-40eb-a635-53baad540bb5 None None] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:52.618106 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97814e9f-ca2a-40eb-a635-53baad540bb5 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:52.618242 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97814e9f-ca2a-40eb-a635-53baad540bb5 None None] PATH_INFO: `/` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:52.619651 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 181/366] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:46:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 11 11:46:52.639471 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d226320-fda8-471d-9478-af1f2257ea9a None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:52.640371 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d226320-fda8-471d-9478-af1f2257ea9a None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:52.641120 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d226320-fda8-471d-9478-af1f2257ea9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:52.661014 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:52.674290 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-3d226320-fda8-471d-9478-af1f2257ea9a None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:46:52.676755 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-3d226320-fda8-471d-9478-af1f2257ea9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:46:52.685547 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74840101-235a-4cf7-9f26-da0ebe8cea3a None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:52.778255 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab176b18-0758-403a-ac41-9a5948f7be65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:52.779531 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab176b18-0758-403a-ac41-9a5948f7be65 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:52.780097 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab176b18-0758-403a-ac41-9a5948f7be65 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:52.780484 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab176b18-0758-403a-ac41-9a5948f7be65 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:52.814255 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 182/367] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:46:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 11 11:46:52.820637 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab176b18-0758-403a-ac41-9a5948f7be65 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:52.828374 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab176b18-0758-403a-ac41-9a5948f7be65 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:52.871724 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application [None req-ab176b18-0758-403a-ac41-9a5948f7be65 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 11:46:52.871724 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:46:52.871724 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:46:52.871724 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:46:52.871724 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:46:52.871724 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:46:52.871724 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:46:52.871724 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:46:52.871724 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:46:52.871724 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:46:52.871724 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:46:52.871724 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:46:52.871724 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 11:46:52.871724 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 11:46:52.871724 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 11:46:52.871724 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 11:46:52.873454 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:46:52.873454 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:46:52.873454 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:46:52.873454 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:46:52.873454 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:46:52.873454 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:46:52.873454 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:46:52.873454 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application with Lock( Oct 11 11:46:52.873454 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:46:52.873454 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:46:52.873454 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:46:52.873454 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:46:52.873454 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:46:52.873454 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:46:52.873454 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:46:52.873454 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:46:52.873454 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 11:46:52.875129 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 11:46:52.875129 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 11:46:52.875129 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:46:52.875129 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 186/368] 10.222.0.59 () {56 vars in 1108 bytes} [Tue Oct 11 11:46:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 419 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 11 11:46:52.885522 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d226320-fda8-471d-9478-af1f2257ea9a None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:52.995177 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:53.000740 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323f3758-c72e-40de-8a74-31be459a9e8d None None] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:53.004121 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323f3758-c72e-40de-8a74-31be459a9e8d None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:53.004121 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323f3758-c72e-40de-8a74-31be459a9e8d None None] PATH_INFO: `/` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:53.004121 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 187/369] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:46:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:46:53.016034 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d336ed-fd9d-470d-a606-d661761d8a88 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:53.016418 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d336ed-fd9d-470d-a606-d661761d8a88 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:53.016620 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d336ed-fd9d-470d-a606-d661761d8a88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:53.051759 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-76d336ed-fd9d-470d-a606-d661761d8a88 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:46:53.053012 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-76d336ed-fd9d-470d-a606-d661761d8a88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:46:53.128482 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d226320-fda8-471d-9478-af1f2257ea9a None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:53.158694 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 188/370] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:46:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 11 11:46:53.160971 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d336ed-fd9d-470d-a606-d661761d8a88 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:53.231105 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5a3a97-1e7d-439c-b709-2118459051dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:53.239419 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5a3a97-1e7d-439c-b709-2118459051dc None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:53.240397 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5a3a97-1e7d-439c-b709-2118459051dc None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:53.241172 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5a3a97-1e7d-439c-b709-2118459051dc None admin] PATH_INFO: `/v3/domains` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:53.242724 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5a3a97-1e7d-439c-b709-2118459051dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:46:53.243654 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5a3a97-1e7d-439c-b709-2118459051dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:53.246631 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5a3a97-1e7d-439c-b709-2118459051dc None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:53.292839 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 183/371] 10.222.0.59 () {56 vars in 1109 bytes} [Tue Oct 11 11:46:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 405 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:46:53.294977 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5a3a97-1e7d-439c-b709-2118459051dc None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:53.351654 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:53.398040 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d336ed-fd9d-470d-a606-d661761d8a88 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:53.445442 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1f51c9-ceb2-465c-b3d4-cebec41e1627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:53.446200 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1f51c9-ceb2-465c-b3d4-cebec41e1627 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:53.446481 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1f51c9-ceb2-465c-b3d4-cebec41e1627 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:53.446793 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1f51c9-ceb2-465c-b3d4-cebec41e1627 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:53.455614 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:53.461538 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1f51c9-ceb2-465c-b3d4-cebec41e1627 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 11:46:53.466845 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5a3a97-1e7d-439c-b709-2118459051dc None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:53.468056 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application [None req-4d1f51c9-ceb2-465c-b3d4-cebec41e1627 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 11 11:46:53.468056 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:46:53.468056 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:46:53.468056 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:46:53.468056 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:46:53.468056 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:46:53.468056 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:46:53.468056 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:46:53.468056 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:46:53.468056 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:46:53.468056 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:46:53.468056 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:46:53.468056 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 11:46:53.468056 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 11:46:53.468056 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 11:46:53.468056 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 11:46:53.469077 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 11:46:53.469077 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 11:46:53.469077 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 11:46:53.469077 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 11:46:53.469077 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:46:53.469077 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:46:53.469077 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:46:53.469077 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:46:53.469077 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:46:53.469077 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:46:53.469077 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:46:53.469077 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application with Lock( Oct 11 11:46:53.469077 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:46:53.469077 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:46:53.469077 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:46:53.469077 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:46:53.469914 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:46:53.469914 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:46:53.469914 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:46:53.469914 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:46:53.469914 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 11:46:53.469914 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 11:46:53.469914 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 11:46:53.469914 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 11:46:53.469914 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 11:46:53.469914 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 11:46:53.469914 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 11 11:46:53.469914 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:46:53.472005 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 189/372] 10.222.0.59 () {56 vars in 1102 bytes} [Tue Oct 11 11:46:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:46:53.473604 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09071f1f-48a4-4326-80b8-a700e706d374 None None] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:53.474074 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09071f1f-48a4-4326-80b8-a700e706d374 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:53.474522 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09071f1f-48a4-4326-80b8-a700e706d374 None None] PATH_INFO: `/` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:53.475992 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 190/373] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:46:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:46:53.478056 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09071f1f-48a4-4326-80b8-a700e706d374 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:53.519399 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f854bf-f4df-453c-9dfa-8245a6a40cde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:53.520602 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f854bf-f4df-453c-9dfa-8245a6a40cde None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:53.521090 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f854bf-f4df-453c-9dfa-8245a6a40cde None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:53.521576 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f854bf-f4df-453c-9dfa-8245a6a40cde None admin] PATH_INFO: `/v3/users/glance` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:53.532404 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f854bf-f4df-453c-9dfa-8245a6a40cde None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:53.533904 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f854bf-f4df-453c-9dfa-8245a6a40cde None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:53.548185 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application [None req-74f854bf-f4df-453c-9dfa-8245a6a40cde None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 11 11:46:53.548185 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:46:53.548185 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:46:53.548185 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:46:53.548185 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:46:53.548185 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:46:53.548185 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:46:53.548185 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:46:53.548185 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:46:53.548185 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:46:53.548185 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:46:53.548185 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:46:53.548185 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 11 11:46:53.548185 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 11 11:46:53.548185 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 11 11:46:53.548185 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 11 11:46:53.550006 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:46:53.550006 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:46:53.550006 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 11 11:46:53.550006 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 11:46:53.550006 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 11 11:46:53.550006 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 11:46:53.550006 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:46:53.550006 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:46:53.550006 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:46:53.550006 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:46:53.550006 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:46:53.550006 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application with Lock( Oct 11 11:46:53.550006 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:46:53.550006 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:46:53.550006 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:46:53.550006 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:46:53.550006 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:46:53.550006 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:46:53.552768 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:46:53.552768 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:46:53.552768 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 11 11:46:53.552768 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 11 11:46:53.552768 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 11 11:46:53.552768 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 11 11:46:53.552768 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 11 11:46:53.552768 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 11 11:46:53.552768 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 11 11:46:53.552768 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:46:53.552768 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 184/374] 10.222.0.59 () {56 vars in 1099 bytes} [Tue Oct 11 11:46:53 2022] GET /identity/v3/users/glance => generated 84 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:46:53.552768 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da4c42c-6bf1-48a4-9722-b6257c608017 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:53.552768 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da4c42c-6bf1-48a4-9722-b6257c608017 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:53.555754 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da4c42c-6bf1-48a4-9722-b6257c608017 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:53.574049 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-3da4c42c-6bf1-48a4-9722-b6257c608017 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:46:53.577279 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-3da4c42c-6bf1-48a4-9722-b6257c608017 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:46:53.601741 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09071f1f-48a4-4326-80b8-a700e706d374 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:53.631476 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:53.642323 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733dc49e-fe5b-4a3d-ae56-82ad49b56d6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:53.643077 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733dc49e-fe5b-4a3d-ae56-82ad49b56d6b None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:53.643358 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733dc49e-fe5b-4a3d-ae56-82ad49b56d6b None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:53.643627 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733dc49e-fe5b-4a3d-ae56-82ad49b56d6b None admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:53.644316 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733dc49e-fe5b-4a3d-ae56-82ad49b56d6b None admin] RBAC: Adding query filter params (name=service) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:46:53.644619 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733dc49e-fe5b-4a3d-ae56-82ad49b56d6b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:53.646070 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733dc49e-fe5b-4a3d-ae56-82ad49b56d6b None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:53.662356 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 191/375] 10.222.0.59 () {56 vars in 1103 bytes} [Tue Oct 11 11:46:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 11:46:53.662356 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733dc49e-fe5b-4a3d-ae56-82ad49b56d6b None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:53.697970 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 185/376] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:46:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 11:46:53.698908 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da4c42c-6bf1-48a4-9722-b6257c608017 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:53.794472 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:53.811409 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733dc49e-fe5b-4a3d-ae56-82ad49b56d6b None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:53.826103 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da4c42c-6bf1-48a4-9722-b6257c608017 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:53.857499 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d306025-37c2-4119-92eb-e31103890beb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:53.858816 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d306025-37c2-4119-92eb-e31103890beb None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:53.859560 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d306025-37c2-4119-92eb-e31103890beb None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:53.860498 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d306025-37c2-4119-92eb-e31103890beb None admin] PATH_INFO: `/v3/roles/service` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:53.868396 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f79d70-2f60-4940-a1e9-659e5dd7caac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:53.869023 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f79d70-2f60-4940-a1e9-659e5dd7caac None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:53.869299 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f79d70-2f60-4940-a1e9-659e5dd7caac None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:53.869564 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f79d70-2f60-4940-a1e9-659e5dd7caac None admin] PATH_INFO: `/v3/users` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:53.870319 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f79d70-2f60-4940-a1e9-659e5dd7caac None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:46:53.870667 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f79d70-2f60-4940-a1e9-659e5dd7caac None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:53.872254 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f79d70-2f60-4940-a1e9-659e5dd7caac None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:53.876306 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d306025-37c2-4119-92eb-e31103890beb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 11:46:53.882313 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application [None req-5d306025-37c2-4119-92eb-e31103890beb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 11 11:46:53.882313 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:46:53.882313 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:46:53.882313 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:46:53.882313 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:46:53.882313 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:46:53.882313 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:46:53.882313 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:46:53.882313 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:46:53.882313 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:46:53.882313 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:46:53.882313 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:46:53.882313 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 11:46:53.882313 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 11:46:53.882313 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 11:46:53.882313 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 11:46:53.883321 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 11:46:53.883321 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 11:46:53.883321 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 11:46:53.883321 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 11:46:53.883321 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:46:53.883321 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:46:53.883321 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:46:53.883321 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:46:53.883321 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:46:53.883321 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:46:53.883321 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:46:53.883321 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application with Lock( Oct 11 11:46:53.883321 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:46:53.883321 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:46:53.883321 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:46:53.883321 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:46:53.884143 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:46:53.884143 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:46:53.884143 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:46:53.884143 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:46:53.884143 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 11:46:53.884143 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 11:46:53.884143 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 11:46:53.884143 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 11:46:53.884143 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 11:46:53.884143 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 11:46:53.884143 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 11 11:46:53.884143 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:46:53.888245 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 192/377] 10.222.0.59 () {56 vars in 1102 bytes} [Tue Oct 11 11:46:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:46:53.889434 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d306025-37c2-4119-92eb-e31103890beb None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:53.923816 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 186/378] 10.222.0.59 () {56 vars in 1137 bytes} [Tue Oct 11 11:46:53 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:46:53.923816 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f79d70-2f60-4940-a1e9-659e5dd7caac None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:53.959184 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:54.065198 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d306025-37c2-4119-92eb-e31103890beb None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:54.065427 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f79d70-2f60-4940-a1e9-659e5dd7caac None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:54.103040 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2aa2a24-26d2-4dad-814e-b2d6057fa2bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:54.103977 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aa2a24-26d2-4dad-814e-b2d6057fa2bc None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:54.104257 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aa2a24-26d2-4dad-814e-b2d6057fa2bc None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:54.104528 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aa2a24-26d2-4dad-814e-b2d6057fa2bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:54.107970 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a17ef7-0f1c-464b-89ba-2b0581406dad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:54.108618 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a17ef7-0f1c-464b-89ba-2b0581406dad None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:54.108883 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a17ef7-0f1c-464b-89ba-2b0581406dad None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:54.109165 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a17ef7-0f1c-464b-89ba-2b0581406dad None admin] PATH_INFO: `/v3/roles` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:54.109841 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a17ef7-0f1c-464b-89ba-2b0581406dad None admin] RBAC: Adding query filter params (name=service) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:46:54.110141 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a17ef7-0f1c-464b-89ba-2b0581406dad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:54.111629 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a17ef7-0f1c-464b-89ba-2b0581406dad None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:54.117895 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:54.118976 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2aa2a24-26d2-4dad-814e-b2d6057fa2bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:54.120796 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2aa2a24-26d2-4dad-814e-b2d6057fa2bc None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:54.126218 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 187/379] 10.222.0.59 () {56 vars in 1103 bytes} [Tue Oct 11 11:46:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:46:54.126861 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a17ef7-0f1c-464b-89ba-2b0581406dad None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:54.150194 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application [None req-c2aa2a24-26d2-4dad-814e-b2d6057fa2bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 11:46:54.150194 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:46:54.150194 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:46:54.150194 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:46:54.150194 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:46:54.150194 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:46:54.150194 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:46:54.150194 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:46:54.150194 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:46:54.150194 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:46:54.150194 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:46:54.150194 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:46:54.150194 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 11:46:54.150194 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 11:46:54.150194 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 11:46:54.150194 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 11:46:54.151669 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:46:54.151669 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:46:54.151669 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:46:54.151669 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:46:54.151669 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:46:54.151669 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:46:54.151669 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:46:54.151669 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application with Lock( Oct 11 11:46:54.151669 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:46:54.151669 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:46:54.151669 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:46:54.151669 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:46:54.151669 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:46:54.151669 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:46:54.151669 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:46:54.151669 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:46:54.151669 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 11:46:54.152728 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 11:46:54.152728 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 11:46:54.152728 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:46:54.158751 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 193/380] 10.222.0.59 () {56 vars in 1108 bytes} [Tue Oct 11 11:46:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:46:54.158751 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2aa2a24-26d2-4dad-814e-b2d6057fa2bc None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:54.255471 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a17ef7-0f1c-464b-89ba-2b0581406dad None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:54.276762 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2aa2a24-26d2-4dad-814e-b2d6057fa2bc None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:54.278821 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:54.319733 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b9ea45-9ccc-4a70-85d5-13bf8ab2b885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:54.320290 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b9ea45-9ccc-4a70-85d5-13bf8ab2b885 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:54.320573 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b9ea45-9ccc-4a70-85d5-13bf8ab2b885 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:54.320904 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b9ea45-9ccc-4a70-85d5-13bf8ab2b885 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:54.322539 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f3b137-3c0b-48b4-8988-6bfa27748e3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:54.323210 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f3b137-3c0b-48b4-8988-6bfa27748e3e None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:54.323485 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f3b137-3c0b-48b4-8988-6bfa27748e3e None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:54.323747 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f3b137-3c0b-48b4-8988-6bfa27748e3e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:54.332640 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b9ea45-9ccc-4a70-85d5-13bf8ab2b885 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 11:46:54.335845 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f3b137-3c0b-48b4-8988-6bfa27748e3e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:54.337329 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f3b137-3c0b-48b4-8988-6bfa27748e3e None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:54.337511 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application [None req-93b9ea45-9ccc-4a70-85d5-13bf8ab2b885 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 11 11:46:54.337511 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:46:54.337511 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:46:54.337511 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:46:54.337511 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:46:54.337511 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:46:54.337511 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:46:54.337511 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:46:54.337511 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:46:54.337511 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:46:54.337511 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:46:54.337511 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:46:54.337511 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 11:46:54.337511 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 11:46:54.337511 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 11:46:54.337511 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 11:46:54.338765 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 11:46:54.338765 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 11:46:54.338765 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 11:46:54.338765 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 11:46:54.338765 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:46:54.338765 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:46:54.338765 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:46:54.338765 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:46:54.338765 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:46:54.338765 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:46:54.338765 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:46:54.338765 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application with Lock( Oct 11 11:46:54.338765 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:46:54.338765 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:46:54.338765 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:46:54.338765 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:46:54.339764 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:46:54.339764 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:46:54.339764 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:46:54.339764 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:46:54.339764 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 11:46:54.339764 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 11:46:54.339764 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 11:46:54.339764 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 11:46:54.339764 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 11:46:54.339764 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 11:46:54.339764 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 11 11:46:54.339764 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:46:54.339764 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 194/381] 10.222.0.59 () {56 vars in 1102 bytes} [Tue Oct 11 11:46:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:46:54.339764 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b9ea45-9ccc-4a70-85d5-13bf8ab2b885 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:54.359556 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application [None req-09f3b137-3c0b-48b4-8988-6bfa27748e3e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 11:46:54.359556 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:46:54.359556 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:46:54.359556 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:46:54.359556 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:46:54.359556 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:46:54.359556 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:46:54.359556 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:46:54.359556 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:46:54.359556 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:46:54.359556 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:46:54.359556 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:46:54.359556 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 11:46:54.359556 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 11:46:54.359556 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 11:46:54.359556 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 11:46:54.360560 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:46:54.360560 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:46:54.360560 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:46:54.360560 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:46:54.360560 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:46:54.360560 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:46:54.360560 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:46:54.360560 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application with Lock( Oct 11 11:46:54.360560 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:46:54.360560 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:46:54.360560 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:46:54.360560 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:46:54.360560 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:46:54.360560 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:46:54.360560 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:46:54.360560 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:46:54.360560 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 11:46:54.361393 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 11:46:54.361393 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 11:46:54.361393 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:46:54.361393 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 188/382] 10.222.0.59 () {56 vars in 1108 bytes} [Tue Oct 11 11:46:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:46:54.361704 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f3b137-3c0b-48b4-8988-6bfa27748e3e None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:54.428283 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:54.478354 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b9ea45-9ccc-4a70-85d5-13bf8ab2b885 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:54.487076 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f3b137-3c0b-48b4-8988-6bfa27748e3e None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:54.514777 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86bea7c-06d4-491b-88e3-c5b796514e0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:54.515421 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86bea7c-06d4-491b-88e3-c5b796514e0d None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:54.515713 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86bea7c-06d4-491b-88e3-c5b796514e0d None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:54.515978 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86bea7c-06d4-491b-88e3-c5b796514e0d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:54.522297 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5ed6d0-018c-42ad-a604-79a5af03869d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:54.522950 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5ed6d0-018c-42ad-a604-79a5af03869d None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:54.523235 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5ed6d0-018c-42ad-a604-79a5af03869d None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:54.523502 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5ed6d0-018c-42ad-a604-79a5af03869d None admin] PATH_INFO: `/v3/domains` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:54.524150 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5ed6d0-018c-42ad-a604-79a5af03869d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:46:54.524446 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5ed6d0-018c-42ad-a604-79a5af03869d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:54.525846 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5ed6d0-018c-42ad-a604-79a5af03869d None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:54.527317 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86bea7c-06d4-491b-88e3-c5b796514e0d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:54.528810 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86bea7c-06d4-491b-88e3-c5b796514e0d None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:54.539234 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 189/383] 10.222.0.59 () {56 vars in 1109 bytes} [Tue Oct 11 11:46:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:46:54.540260 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5ed6d0-018c-42ad-a604-79a5af03869d None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:54.547011 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application [None req-d86bea7c-06d4-491b-88e3-c5b796514e0d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 11:46:54.547011 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:46:54.547011 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:46:54.547011 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:46:54.547011 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:46:54.547011 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:46:54.547011 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:46:54.547011 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:46:54.547011 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:46:54.547011 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:46:54.547011 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:46:54.547011 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:46:54.547011 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 11:46:54.547011 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 11:46:54.547011 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 11:46:54.547011 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 11:46:54.548076 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:46:54.548076 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:46:54.548076 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:46:54.548076 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:46:54.548076 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:46:54.548076 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:46:54.548076 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:46:54.548076 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application with Lock( Oct 11 11:46:54.548076 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:46:54.548076 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:46:54.548076 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:46:54.548076 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:46:54.548076 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:46:54.548076 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:46:54.548076 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:46:54.548076 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:46:54.548076 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 11:46:54.553473 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 11:46:54.553473 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 11:46:54.553473 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:46:54.553473 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 195/384] 10.222.0.59 () {56 vars in 1108 bytes} [Tue Oct 11 11:46:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:46:54.553473 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86bea7c-06d4-491b-88e3-c5b796514e0d None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:54.573748 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:54.661175 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5ed6d0-018c-42ad-a604-79a5af03869d None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:54.675260 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86bea7c-06d4-491b-88e3-c5b796514e0d None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:54.680186 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:54.703862 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccc56b4-996b-4917-b882-448ce725ffa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:54.704490 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccc56b4-996b-4917-b882-448ce725ffa1 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:54.704770 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccc56b4-996b-4917-b882-448ce725ffa1 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:54.705049 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccc56b4-996b-4917-b882-448ce725ffa1 None admin] PATH_INFO: `/v3/roles` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:54.705723 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccc56b4-996b-4917-b882-448ce725ffa1 None admin] RBAC: Adding query filter params (name=service) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:46:54.706061 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccc56b4-996b-4917-b882-448ce725ffa1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:54.707554 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccc56b4-996b-4917-b882-448ce725ffa1 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:54.710237 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14072c71-86a5-486e-ae17-f1036dfc4897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:54.710869 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14072c71-86a5-486e-ae17-f1036dfc4897 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:54.711169 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14072c71-86a5-486e-ae17-f1036dfc4897 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:54.711440 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14072c71-86a5-486e-ae17-f1036dfc4897 None admin] PATH_INFO: `/v3/domains` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:54.712068 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14072c71-86a5-486e-ae17-f1036dfc4897 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:46:54.712366 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14072c71-86a5-486e-ae17-f1036dfc4897 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:54.713772 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14072c71-86a5-486e-ae17-f1036dfc4897 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:54.716385 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 190/385] 10.222.0.59 () {56 vars in 1103 bytes} [Tue Oct 11 11:46:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:46:54.719987 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e649a392-9495-4a07-83b4-bdc6087c0cb5 None None] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:54.720283 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e649a392-9495-4a07-83b4-bdc6087c0cb5 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:54.720582 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e649a392-9495-4a07-83b4-bdc6087c0cb5 None None] PATH_INFO: `/` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:54.722122 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 191/386] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:46:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:46:54.722918 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e649a392-9495-4a07-83b4-bdc6087c0cb5 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:54.726927 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 196/387] 10.222.0.59 () {56 vars in 1109 bytes} [Tue Oct 11 11:46:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:46:54.727794 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14072c71-86a5-486e-ae17-f1036dfc4897 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:54.805203 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:54.833983 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14072c71-86a5-486e-ae17-f1036dfc4897 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:54.841786 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e649a392-9495-4a07-83b4-bdc6087c0cb5 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:54.873428 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb72c06-de5b-411c-adc7-ed587968fdf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:54.874046 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb72c06-de5b-411c-adc7-ed587968fdf8 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:54.874328 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb72c06-de5b-411c-adc7-ed587968fdf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:54.874656 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb72c06-de5b-411c-adc7-ed587968fdf8 None admin] PATH_INFO: `/v3/domains` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:54.875324 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb72c06-de5b-411c-adc7-ed587968fdf8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:46:54.875630 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb72c06-de5b-411c-adc7-ed587968fdf8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:54.877039 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb72c06-de5b-411c-adc7-ed587968fdf8 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:54.877604 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139500fc-61a3-4542-8f0f-cd4e0d9fa95f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:54.878246 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139500fc-61a3-4542-8f0f-cd4e0d9fa95f None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:54.878516 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139500fc-61a3-4542-8f0f-cd4e0d9fa95f None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:54.878818 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139500fc-61a3-4542-8f0f-cd4e0d9fa95f None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:54.886864 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139500fc-61a3-4542-8f0f-cd4e0d9fa95f None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:54.888318 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139500fc-61a3-4542-8f0f-cd4e0d9fa95f None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:54.890684 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 197/388] 10.222.0.59 () {56 vars in 1109 bytes} [Tue Oct 11 11:46:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:46:54.891713 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb72c06-de5b-411c-adc7-ed587968fdf8 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:54.901726 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application [None req-139500fc-61a3-4542-8f0f-cd4e0d9fa95f None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 11 11:46:54.901726 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:46:54.901726 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:46:54.901726 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:46:54.901726 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:46:54.901726 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:46:54.901726 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:46:54.901726 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:46:54.901726 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:46:54.901726 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:46:54.901726 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:46:54.901726 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:46:54.901726 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 11 11:46:54.901726 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 11 11:46:54.901726 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 11 11:46:54.901726 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 11 11:46:54.903119 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:46:54.903119 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:46:54.903119 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 11 11:46:54.903119 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 11:46:54.903119 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 11 11:46:54.903119 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 11:46:54.903119 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:46:54.903119 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:46:54.903119 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:46:54.903119 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:46:54.903119 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:46:54.903119 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application with Lock( Oct 11 11:46:54.903119 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:46:54.903119 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:46:54.903119 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:46:54.903119 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:46:54.903119 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:46:54.903119 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:46:54.903990 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:46:54.903990 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:46:54.903990 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 11 11:46:54.903990 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 11 11:46:54.903990 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 11 11:46:54.903990 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 11 11:46:54.903990 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 11 11:46:54.903990 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 11 11:46:54.903990 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 11 11:46:54.903990 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:46:54.903990 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 192/389] 10.222.0.59 () {56 vars in 1102 bytes} [Tue Oct 11 11:46:54 2022] GET /identity/v3/users/neutron => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:46:54.904549 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba236d2-a92d-49f4-a14a-b1d9dc5734cd None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:54.904549 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba236d2-a92d-49f4-a14a-b1d9dc5734cd None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:54.904663 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba236d2-a92d-49f4-a14a-b1d9dc5734cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:54.931452 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-1ba236d2-a92d-49f4-a14a-b1d9dc5734cd None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:46:54.933515 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-1ba236d2-a92d-49f4-a14a-b1d9dc5734cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:46:54.973467 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:55.010899 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb72c06-de5b-411c-adc7-ed587968fdf8 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:55.021844 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 193/390] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:46:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 11:46:55.022832 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba236d2-a92d-49f4-a14a-b1d9dc5734cd None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:55.066848 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd27c31f-bbcf-4bd4-b996-3bc7733cddf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:55.067497 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd27c31f-bbcf-4bd4-b996-3bc7733cddf6 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:55.067766 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd27c31f-bbcf-4bd4-b996-3bc7733cddf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:55.068038 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd27c31f-bbcf-4bd4-b996-3bc7733cddf6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:55.071584 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:55.082129 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd27c31f-bbcf-4bd4-b996-3bc7733cddf6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:55.083661 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd27c31f-bbcf-4bd4-b996-3bc7733cddf6 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:55.108733 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application [None req-cd27c31f-bbcf-4bd4-b996-3bc7733cddf6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 11:46:55.108733 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:46:55.108733 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:46:55.108733 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:46:55.108733 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:46:55.108733 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:46:55.108733 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:46:55.108733 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:46:55.108733 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:46:55.108733 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:46:55.108733 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:46:55.108733 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:46:55.108733 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 11:46:55.108733 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 11:46:55.108733 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 11:46:55.108733 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 11:46:55.109760 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:46:55.109760 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:46:55.109760 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:46:55.109760 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:46:55.109760 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:46:55.109760 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:46:55.109760 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:46:55.109760 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application with Lock( Oct 11 11:46:55.109760 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:46:55.109760 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:46:55.109760 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:46:55.109760 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:46:55.109760 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:46:55.109760 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:46:55.109760 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:46:55.109760 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:46:55.109760 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 11:46:55.110634 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 11:46:55.110634 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 11:46:55.110634 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:46:55.110822 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 198/391] 10.222.0.59 () {56 vars in 1108 bytes} [Tue Oct 11 11:46:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:46:55.113382 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd27c31f-bbcf-4bd4-b996-3bc7733cddf6 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:55.152848 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba236d2-a92d-49f4-a14a-b1d9dc5734cd None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:55.207272 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fecc7c-9d06-48c7-991b-6531419051bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:55.208456 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fecc7c-9d06-48c7-991b-6531419051bb None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:55.208456 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fecc7c-9d06-48c7-991b-6531419051bb None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:55.208625 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fecc7c-9d06-48c7-991b-6531419051bb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:55.221222 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fecc7c-9d06-48c7-991b-6531419051bb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:55.224106 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fecc7c-9d06-48c7-991b-6531419051bb None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:55.246968 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application [None req-16fecc7c-9d06-48c7-991b-6531419051bb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 11:46:55.246968 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:46:55.246968 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:46:55.246968 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:46:55.246968 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:46:55.246968 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:46:55.246968 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:46:55.246968 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:46:55.246968 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:46:55.246968 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:46:55.246968 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:46:55.246968 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:46:55.246968 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 11 11:46:55.246968 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 11 11:46:55.246968 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 11 11:46:55.247973 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 11 11:46:55.247973 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:46:55.247973 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:46:55.247973 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:46:55.247973 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:46:55.247973 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:46:55.247973 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:46:55.247973 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:46:55.247973 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application with Lock( Oct 11 11:46:55.247973 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:46:55.247973 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:46:55.247973 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:46:55.247973 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:46:55.247973 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:46:55.247973 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:46:55.247973 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:46:55.247973 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:46:55.248916 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 11 11:46:55.248916 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 11 11:46:55.248916 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 11 11:46:55.248916 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 11 11:46:55.248916 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 11 11:46:55.248916 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 11 11:46:55.248916 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 11:46:55.248916 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:46:55.250033 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 194/392] 10.222.0.59 () {56 vars in 1111 bytes} [Tue Oct 11 11:46:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:46:55.251515 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fecc7c-9d06-48c7-991b-6531419051bb None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:55.257181 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd27c31f-bbcf-4bd4-b996-3bc7733cddf6 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:55.302033 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:55.314146 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6fad67-805b-47b9-954c-d36ba473e754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:55.314931 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6fad67-805b-47b9-954c-d36ba473e754 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:55.315215 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6fad67-805b-47b9-954c-d36ba473e754 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:55.315495 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6fad67-805b-47b9-954c-d36ba473e754 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:55.324986 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6fad67-805b-47b9-954c-d36ba473e754 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:55.326575 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6fad67-805b-47b9-954c-d36ba473e754 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:55.340981 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application [None req-9e6fad67-805b-47b9-954c-d36ba473e754 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 11 11:46:55.340981 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:46:55.340981 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:46:55.340981 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:46:55.340981 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:46:55.340981 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:46:55.340981 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:46:55.340981 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:46:55.340981 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:46:55.340981 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:46:55.340981 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:46:55.340981 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:46:55.340981 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 11 11:46:55.340981 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 11 11:46:55.340981 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 11 11:46:55.340981 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 11 11:46:55.342772 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:46:55.342772 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:46:55.342772 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 11 11:46:55.342772 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 11:46:55.342772 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 11 11:46:55.342772 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 11:46:55.342772 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:46:55.342772 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:46:55.342772 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:46:55.342772 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:46:55.342772 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:46:55.342772 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application with Lock( Oct 11 11:46:55.342772 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:46:55.342772 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:46:55.342772 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:46:55.342772 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:46:55.342772 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:46:55.342772 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:46:55.345300 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:46:55.345300 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:46:55.345300 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 11 11:46:55.345300 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 11 11:46:55.345300 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 11 11:46:55.345300 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 11 11:46:55.345300 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 11 11:46:55.345300 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 11 11:46:55.345300 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 11 11:46:55.345300 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:46:55.345300 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 199/393] 10.222.0.59 () {56 vars in 1093 bytes} [Tue Oct 11 11:46:55 2022] GET /identity/v3/users/nova => generated 82 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:46:55.348307 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6fad67-805b-47b9-954c-d36ba473e754 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:55.398741 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fecc7c-9d06-48c7-991b-6531419051bb None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:55.401143 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:55.437730 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a9bcf2-8866-4e1e-9036-a9b106ab0405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:55.438511 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a9bcf2-8866-4e1e-9036-a9b106ab0405 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:55.438930 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a9bcf2-8866-4e1e-9036-a9b106ab0405 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:55.439325 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a9bcf2-8866-4e1e-9036-a9b106ab0405 None admin] PATH_INFO: `/v3/users` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:55.440314 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a9bcf2-8866-4e1e-9036-a9b106ab0405 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:46:55.440719 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a9bcf2-8866-4e1e-9036-a9b106ab0405 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:55.442387 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a9bcf2-8866-4e1e-9036-a9b106ab0405 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:55.461684 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 195/394] 10.222.0.59 () {56 vars in 1139 bytes} [Tue Oct 11 11:46:55 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:46:55.462586 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a9bcf2-8866-4e1e-9036-a9b106ab0405 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:55.478465 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6fad67-805b-47b9-954c-d36ba473e754 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:55.533772 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678ad47a-59d9-4a91-ba01-5d00a51aa275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:55.535089 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678ad47a-59d9-4a91-ba01-5d00a51aa275 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:55.535685 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678ad47a-59d9-4a91-ba01-5d00a51aa275 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:55.535819 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678ad47a-59d9-4a91-ba01-5d00a51aa275 None admin] PATH_INFO: `/v3/domains` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:55.536684 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678ad47a-59d9-4a91-ba01-5d00a51aa275 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:46:55.536790 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678ad47a-59d9-4a91-ba01-5d00a51aa275 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:55.538140 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678ad47a-59d9-4a91-ba01-5d00a51aa275 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:55.552997 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 200/395] 10.222.0.59 () {56 vars in 1109 bytes} [Tue Oct 11 11:46:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:46:55.553898 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678ad47a-59d9-4a91-ba01-5d00a51aa275 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:55.560687 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:55.592720 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a9bcf2-8866-4e1e-9036-a9b106ab0405 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:55.635251 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53de0abc-d725-4529-bc84-983a1f0ad2f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:55.636326 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53de0abc-d725-4529-bc84-983a1f0ad2f0 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:55.636829 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53de0abc-d725-4529-bc84-983a1f0ad2f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:55.637348 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53de0abc-d725-4529-bc84-983a1f0ad2f0 None admin] PATH_INFO: `/v3/projects` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:55.638701 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53de0abc-d725-4529-bc84-983a1f0ad2f0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:46:55.639238 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53de0abc-d725-4529-bc84-983a1f0ad2f0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:55.641987 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53de0abc-d725-4529-bc84-983a1f0ad2f0 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:55.657502 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678ad47a-59d9-4a91-ba01-5d00a51aa275 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:55.665985 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:55.676353 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 196/396] 10.222.0.59 () {56 vars in 1148 bytes} [Tue Oct 11 11:46:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:46:55.677901 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53de0abc-d725-4529-bc84-983a1f0ad2f0 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:55.697794 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241224ca-ccbe-481d-8f9a-c004b537ee1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:55.698598 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241224ca-ccbe-481d-8f9a-c004b537ee1b None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:55.699016 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241224ca-ccbe-481d-8f9a-c004b537ee1b None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:55.699408 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241224ca-ccbe-481d-8f9a-c004b537ee1b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:55.711191 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241224ca-ccbe-481d-8f9a-c004b537ee1b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:55.712722 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241224ca-ccbe-481d-8f9a-c004b537ee1b None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:55.729161 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application [None req-241224ca-ccbe-481d-8f9a-c004b537ee1b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 11:46:55.729161 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:46:55.729161 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:46:55.729161 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:46:55.729161 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:46:55.729161 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:46:55.729161 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:46:55.729161 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:46:55.729161 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:46:55.729161 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:46:55.729161 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:46:55.729161 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:46:55.729161 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 11:46:55.729161 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 11:46:55.729161 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 11:46:55.729161 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 11:46:55.730511 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:46:55.730511 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:46:55.730511 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:46:55.730511 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:46:55.730511 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:46:55.730511 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:46:55.730511 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:46:55.730511 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application with Lock( Oct 11 11:46:55.730511 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:46:55.730511 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:46:55.730511 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:46:55.730511 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:46:55.730511 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:46:55.730511 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:46:55.730511 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:46:55.730511 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:46:55.730511 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 11:46:55.732166 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 11:46:55.732166 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 11:46:55.732166 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:46:55.732166 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 201/397] 10.222.0.59 () {56 vars in 1108 bytes} [Tue Oct 11 11:46:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:46:55.732166 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241224ca-ccbe-481d-8f9a-c004b537ee1b None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:55.808726 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53de0abc-d725-4529-bc84-983a1f0ad2f0 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:55.840739 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241224ca-ccbe-481d-8f9a-c004b537ee1b None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:55.862688 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b758772-3756-4b0a-a86e-e6506ab2ac21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:55.863310 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b758772-3756-4b0a-a86e-e6506ab2ac21 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:55.863586 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b758772-3756-4b0a-a86e-e6506ab2ac21 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:55.863853 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b758772-3756-4b0a-a86e-e6506ab2ac21 None admin] PATH_INFO: `/v3/users` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:55.864584 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b758772-3756-4b0a-a86e-e6506ab2ac21 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:46:55.864884 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b758772-3756-4b0a-a86e-e6506ab2ac21 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:55.866340 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b758772-3756-4b0a-a86e-e6506ab2ac21 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:55.881373 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf58f3ca-57d5-4372-aa7b-87e3e3951837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:55.882021 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf58f3ca-57d5-4372-aa7b-87e3e3951837 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:55.882301 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf58f3ca-57d5-4372-aa7b-87e3e3951837 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:55.882606 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf58f3ca-57d5-4372-aa7b-87e3e3951837 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:55.883034 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:55.887332 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 197/398] 10.222.0.59 () {56 vars in 1133 bytes} [Tue Oct 11 11:46:55 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:46:55.888223 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b758772-3756-4b0a-a86e-e6506ab2ac21 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:55.895671 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf58f3ca-57d5-4372-aa7b-87e3e3951837 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:55.897204 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf58f3ca-57d5-4372-aa7b-87e3e3951837 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:55.915809 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application [None req-cf58f3ca-57d5-4372-aa7b-87e3e3951837 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 11:46:55.915809 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:46:55.915809 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:46:55.915809 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:46:55.915809 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:46:55.915809 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:46:55.915809 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:46:55.915809 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:46:55.915809 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:46:55.915809 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:46:55.915809 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:46:55.915809 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:46:55.915809 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 11:46:55.915809 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 11:46:55.915809 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 11:46:55.915809 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 11:46:55.917545 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:46:55.917545 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:46:55.917545 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:46:55.917545 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:46:55.917545 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:46:55.917545 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:46:55.917545 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:46:55.917545 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application with Lock( Oct 11 11:46:55.917545 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:46:55.917545 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:46:55.917545 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:46:55.917545 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:46:55.917545 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:46:55.917545 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:46:55.917545 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:46:55.917545 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:46:55.917545 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 11:46:55.919198 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 11:46:55.919198 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 11:46:55.919198 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:46:55.919198 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 202/399] 10.222.0.59 () {56 vars in 1108 bytes} [Tue Oct 11 11:46:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:46:55.919198 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf58f3ca-57d5-4372-aa7b-87e3e3951837 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:55.936325 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:56.008102 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b758772-3756-4b0a-a86e-e6506ab2ac21 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:56.048558 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a86e91-880e-4036-a7f6-c73f3a841481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:56.049242 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a86e91-880e-4036-a7f6-c73f3a841481 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:56.050840 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a86e91-880e-4036-a7f6-c73f3a841481 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:56.051734 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a86e91-880e-4036-a7f6-c73f3a841481 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:56.061544 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a86e91-880e-4036-a7f6-c73f3a841481 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:56.063073 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a86e91-880e-4036-a7f6-c73f3a841481 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:56.067347 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf58f3ca-57d5-4372-aa7b-87e3e3951837 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:56.077999 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application [None req-00a86e91-880e-4036-a7f6-c73f3a841481 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 11 11:46:56.077999 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:46:56.077999 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:46:56.077999 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:46:56.077999 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:46:56.077999 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:46:56.077999 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:46:56.077999 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:46:56.077999 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:46:56.077999 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:46:56.077999 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:46:56.077999 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:46:56.077999 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 11 11:46:56.077999 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 11 11:46:56.077999 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 11 11:46:56.077999 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 11 11:46:56.079033 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:46:56.079033 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:46:56.079033 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 11 11:46:56.079033 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 11:46:56.079033 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 11 11:46:56.079033 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 11:46:56.079033 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:46:56.079033 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:46:56.079033 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:46:56.079033 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:46:56.079033 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:46:56.079033 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application with Lock( Oct 11 11:46:56.079033 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:46:56.079033 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:46:56.079033 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:46:56.079033 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:46:56.079033 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:46:56.079033 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:46:56.079906 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:46:56.079906 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:46:56.079906 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 11 11:46:56.079906 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 11 11:46:56.079906 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 11 11:46:56.079906 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 11 11:46:56.079906 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 11 11:46:56.079906 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 11 11:46:56.079906 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 11 11:46:56.079906 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:46:56.079906 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 198/400] 10.222.0.59 () {56 vars in 1099 bytes} [Tue Oct 11 11:46:55 2022] GET /identity/v3/users/cinder => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:46:56.080515 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a86e91-880e-4036-a7f6-c73f3a841481 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:56.107536 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e11921-a4f3-4bc1-8e21-3539a73c877b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:56.108211 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e11921-a4f3-4bc1-8e21-3539a73c877b None admin] REQUEST_METHOD: `PUT` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:56.108492 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e11921-a4f3-4bc1-8e21-3539a73c877b None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:56.108767 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e11921-a4f3-4bc1-8e21-3539a73c877b None admin] PATH_INFO: `/v3/projects/26ecf552cc9a4e5ca1d380509683dc4a/users/91a00658b79a42bd91a22a2508b5de34/roles/f5a43d8f3fa24a87a7abe7492cf6f87f` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:56.126269 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e11921-a4f3-4bc1-8e21-3539a73c877b None admin] RBAC: Authorizing `identity:create_grant(project_id=26ecf552cc9a4e5ca1d380509683dc4a, user_id=91a00658b79a42bd91a22a2508b5de34, role_id=f5a43d8f3fa24a87a7abe7492cf6f87f)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:56.127883 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e11921-a4f3-4bc1-8e21-3539a73c877b None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:56.142053 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 203/401] 10.222.0.59 () {58 vars in 1439 bytes} [Tue Oct 11 11:46:55 2022] PUT /identity/v3/projects/26ecf552cc9a4e5ca1d380509683dc4a/users/91a00658b79a42bd91a22a2508b5de34/roles/f5a43d8f3fa24a87a7abe7492cf6f87f => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 11:46:56.142940 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e11921-a4f3-4bc1-8e21-3539a73c877b None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:56.166828 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:56.229975 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a86e91-880e-4036-a7f6-c73f3a841481 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:56.266758 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:56.291484 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa1c988-29a6-4dee-8cd9-9e89b9aa29a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:56.292365 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa1c988-29a6-4dee-8cd9-9e89b9aa29a3 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:56.292838 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa1c988-29a6-4dee-8cd9-9e89b9aa29a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:56.293302 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa1c988-29a6-4dee-8cd9-9e89b9aa29a3 None admin] PATH_INFO: `/v3/domains` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:56.294456 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa1c988-29a6-4dee-8cd9-9e89b9aa29a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:46:56.295019 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa1c988-29a6-4dee-8cd9-9e89b9aa29a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:56.297586 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa1c988-29a6-4dee-8cd9-9e89b9aa29a3 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:56.318961 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 199/402] 10.222.0.59 () {56 vars in 1109 bytes} [Tue Oct 11 11:46:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:46:56.320706 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa1c988-29a6-4dee-8cd9-9e89b9aa29a3 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:56.331836 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e11921-a4f3-4bc1-8e21-3539a73c877b None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:56.379839 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f7a01e-3b6a-4773-b6c4-48298d228a6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:56.380510 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f7a01e-3b6a-4773-b6c4-48298d228a6a None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:56.380787 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f7a01e-3b6a-4773-b6c4-48298d228a6a None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:56.381057 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f7a01e-3b6a-4773-b6c4-48298d228a6a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:56.394646 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f7a01e-3b6a-4773-b6c4-48298d228a6a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:56.396189 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f7a01e-3b6a-4773-b6c4-48298d228a6a None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:56.414929 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application [None req-48f7a01e-3b6a-4773-b6c4-48298d228a6a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 11:46:56.414929 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:46:56.414929 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:46:56.414929 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:46:56.414929 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:46:56.414929 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:46:56.414929 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:46:56.414929 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:46:56.414929 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:46:56.414929 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:46:56.414929 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:46:56.414929 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:46:56.414929 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 11:46:56.414929 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 11:46:56.414929 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 11:46:56.414929 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 11:46:56.416516 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:46:56.416516 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:46:56.416516 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:46:56.416516 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:46:56.416516 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:46:56.416516 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:46:56.416516 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:46:56.416516 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application with Lock( Oct 11 11:46:56.416516 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:46:56.416516 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:46:56.416516 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:46:56.416516 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:46:56.416516 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:46:56.416516 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:46:56.416516 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:46:56.416516 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:46:56.416516 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 11:46:56.418046 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 11:46:56.418046 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 11:46:56.418046 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:46:56.418046 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 204/403] 10.222.0.59 () {56 vars in 1108 bytes} [Tue Oct 11 11:46:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:46:56.418046 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f7a01e-3b6a-4773-b6c4-48298d228a6a None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:56.504006 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa1c988-29a6-4dee-8cd9-9e89b9aa29a3 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:56.542213 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:56.566526 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc2c1cb-a122-4aaf-9d61-d759a1fb1816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:56.567588 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc2c1cb-a122-4aaf-9d61-d759a1fb1816 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:56.567981 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc2c1cb-a122-4aaf-9d61-d759a1fb1816 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:56.568401 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc2c1cb-a122-4aaf-9d61-d759a1fb1816 None admin] PATH_INFO: `/v3/domains` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:56.569480 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc2c1cb-a122-4aaf-9d61-d759a1fb1816 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:46:56.569979 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc2c1cb-a122-4aaf-9d61-d759a1fb1816 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:56.571985 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f7a01e-3b6a-4773-b6c4-48298d228a6a None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:56.572464 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc2c1cb-a122-4aaf-9d61-d759a1fb1816 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:56.591423 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:56.592615 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 200/404] 10.222.0.59 () {56 vars in 1109 bytes} [Tue Oct 11 11:46:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:46:56.598414 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc2c1cb-a122-4aaf-9d61-d759a1fb1816 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:56.632158 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ee6439-9875-473a-8a32-0e2a267b856e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:56.633146 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ee6439-9875-473a-8a32-0e2a267b856e None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:56.633597 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ee6439-9875-473a-8a32-0e2a267b856e None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:56.634046 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ee6439-9875-473a-8a32-0e2a267b856e None admin] PATH_INFO: `/v3/users` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:56.635382 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ee6439-9875-473a-8a32-0e2a267b856e None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:46:56.635895 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ee6439-9875-473a-8a32-0e2a267b856e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:56.638496 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ee6439-9875-473a-8a32-0e2a267b856e None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:56.714055 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 205/405] 10.222.0.59 () {56 vars in 1137 bytes} [Tue Oct 11 11:46:56 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:46:56.714055 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ee6439-9875-473a-8a32-0e2a267b856e None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:56.738439 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc2c1cb-a122-4aaf-9d61-d759a1fb1816 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:56.794278 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1e1c9b-d6d2-4c76-8c06-221298523cb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:56.795386 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1e1c9b-d6d2-4c76-8c06-221298523cb0 None admin] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:56.795883 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1e1c9b-d6d2-4c76-8c06-221298523cb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:56.796340 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1e1c9b-d6d2-4c76-8c06-221298523cb0 None admin] PATH_INFO: `/v3/users` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:56.797532 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1e1c9b-d6d2-4c76-8c06-221298523cb0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:56.800400 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1e1c9b-d6d2-4c76-8c06-221298523cb0 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:56.838663 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 201/406] 10.222.0.59 () {60 vars in 1132 bytes} [Tue Oct 11 11:46:56 2022] POST /identity/v3/users => generated 247 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 11:46:56.839588 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1e1c9b-d6d2-4c76-8c06-221298523cb0 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:56.865333 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ee6439-9875-473a-8a32-0e2a267b856e None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:56.908515 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:56.944566 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca03e79-fd9c-450f-8cb1-3f83afac669f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:56.945649 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca03e79-fd9c-450f-8cb1-3f83afac669f None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:56.946122 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca03e79-fd9c-450f-8cb1-3f83afac669f None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:56.946629 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca03e79-fd9c-450f-8cb1-3f83afac669f None admin] PATH_INFO: `/v3/domains` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:56.947845 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca03e79-fd9c-450f-8cb1-3f83afac669f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:46:56.948371 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca03e79-fd9c-450f-8cb1-3f83afac669f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:56.950971 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca03e79-fd9c-450f-8cb1-3f83afac669f None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:56.978607 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 206/407] 10.222.0.59 () {56 vars in 1109 bytes} [Tue Oct 11 11:46:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:46:56.978607 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca03e79-fd9c-450f-8cb1-3f83afac669f None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:57.032669 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:57.069561 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1e1c9b-d6d2-4c76-8c06-221298523cb0 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:57.119250 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6cda64-be17-4cd2-8c5d-defcc32c5094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:57.120143 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6cda64-be17-4cd2-8c5d-defcc32c5094 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:57.120431 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6cda64-be17-4cd2-8c5d-defcc32c5094 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:57.120735 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6cda64-be17-4cd2-8c5d-defcc32c5094 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:57.129151 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6cda64-be17-4cd2-8c5d-defcc32c5094 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:57.130882 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6cda64-be17-4cd2-8c5d-defcc32c5094 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:57.145612 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application [None req-0a6cda64-be17-4cd2-8c5d-defcc32c5094 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 11:46:57.145612 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:46:57.145612 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:46:57.145612 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:46:57.145612 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:46:57.145612 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:46:57.145612 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:46:57.145612 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:46:57.145612 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:46:57.145612 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:46:57.145612 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:46:57.145612 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:46:57.145612 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 11 11:46:57.145612 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 11 11:46:57.145612 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 11 11:46:57.147287 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 11 11:46:57.147287 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:46:57.147287 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:46:57.147287 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:46:57.147287 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:46:57.147287 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:46:57.147287 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:46:57.147287 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:46:57.147287 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application with Lock( Oct 11 11:46:57.147287 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:46:57.147287 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:46:57.147287 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:46:57.147287 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:46:57.147287 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:46:57.147287 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:46:57.147287 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:46:57.147287 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:46:57.149134 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 11 11:46:57.149134 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 11 11:46:57.149134 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 11 11:46:57.149134 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 11 11:46:57.149134 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 11 11:46:57.149134 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 11 11:46:57.149134 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 11:46:57.149134 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:46:57.152002 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 202/408] 10.222.0.59 () {56 vars in 1111 bytes} [Tue Oct 11 11:46:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:46:57.153057 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6cda64-be17-4cd2-8c5d-defcc32c5094 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:57.179086 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca03e79-fd9c-450f-8cb1-3f83afac669f None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:57.243292 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883af04f-a69d-43e6-803d-319dd488b5b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:57.243879 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883af04f-a69d-43e6-803d-319dd488b5b5 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:57.244162 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883af04f-a69d-43e6-803d-319dd488b5b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:57.244524 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883af04f-a69d-43e6-803d-319dd488b5b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:57.257357 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883af04f-a69d-43e6-803d-319dd488b5b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:57.260988 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883af04f-a69d-43e6-803d-319dd488b5b5 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:57.280136 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application [None req-883af04f-a69d-43e6-803d-319dd488b5b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 11:46:57.280136 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:46:57.280136 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:46:57.280136 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:46:57.280136 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:46:57.280136 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:46:57.280136 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:46:57.280136 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:46:57.280136 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:46:57.280136 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:46:57.280136 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:46:57.280136 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:46:57.280136 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 11:46:57.280136 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 11:46:57.280136 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 11:46:57.280136 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 11:46:57.282357 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:46:57.282357 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:46:57.282357 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:46:57.282357 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:46:57.282357 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:46:57.282357 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:46:57.282357 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:46:57.282357 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application with Lock( Oct 11 11:46:57.282357 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:46:57.282357 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:46:57.282357 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:46:57.282357 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:46:57.282357 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:46:57.282357 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:46:57.282357 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:46:57.282357 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:46:57.282357 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 11:46:57.283968 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 11:46:57.283968 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 11:46:57.283968 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:46:57.283968 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 207/409] 10.222.0.59 () {56 vars in 1108 bytes} [Tue Oct 11 11:46:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:46:57.283968 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883af04f-a69d-43e6-803d-319dd488b5b5 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:57.320107 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:57.331691 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6cda64-be17-4cd2-8c5d-defcc32c5094 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:57.386827 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:57.397396 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e7c723-8b40-408a-bfb1-1f6b7905b371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:57.398533 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e7c723-8b40-408a-bfb1-1f6b7905b371 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:57.399174 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e7c723-8b40-408a-bfb1-1f6b7905b371 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:57.399543 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e7c723-8b40-408a-bfb1-1f6b7905b371 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:57.411094 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883af04f-a69d-43e6-803d-319dd488b5b5 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:57.413131 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e7c723-8b40-408a-bfb1-1f6b7905b371 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:57.416099 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e7c723-8b40-408a-bfb1-1f6b7905b371 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:57.439322 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application [None req-49e7c723-8b40-408a-bfb1-1f6b7905b371 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 11:46:57.439322 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:46:57.439322 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:46:57.439322 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:46:57.439322 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:46:57.439322 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:46:57.439322 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:46:57.439322 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:46:57.439322 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:46:57.439322 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:46:57.439322 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:46:57.439322 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:46:57.439322 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 11 11:46:57.439322 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 11 11:46:57.439322 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 11 11:46:57.440391 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 11 11:46:57.440391 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:46:57.440391 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:46:57.440391 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:46:57.440391 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:46:57.440391 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:46:57.440391 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:46:57.440391 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:46:57.440391 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application with Lock( Oct 11 11:46:57.440391 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:46:57.440391 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:46:57.440391 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:46:57.440391 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:46:57.440391 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:46:57.440391 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:46:57.440391 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:46:57.440391 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:46:57.441370 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 11 11:46:57.441370 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 11 11:46:57.441370 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 11 11:46:57.441370 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 11 11:46:57.441370 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 11 11:46:57.441370 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 11 11:46:57.441370 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 11:46:57.441370 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:46:57.441794 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 203/410] 10.222.0.59 () {56 vars in 1111 bytes} [Tue Oct 11 11:46:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:46:57.443586 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e7c723-8b40-408a-bfb1-1f6b7905b371 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:57.455977 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bc6dff-fb04-4c12-b0a3-45913f124a28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:57.457006 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bc6dff-fb04-4c12-b0a3-45913f124a28 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:57.457493 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bc6dff-fb04-4c12-b0a3-45913f124a28 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:57.457958 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bc6dff-fb04-4c12-b0a3-45913f124a28 None admin] PATH_INFO: `/v3/projects` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:57.458902 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bc6dff-fb04-4c12-b0a3-45913f124a28 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:46:57.459433 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bc6dff-fb04-4c12-b0a3-45913f124a28 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:57.461396 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bc6dff-fb04-4c12-b0a3-45913f124a28 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:57.490411 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 208/411] 10.222.0.59 () {56 vars in 1148 bytes} [Tue Oct 11 11:46:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:46:57.491883 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bc6dff-fb04-4c12-b0a3-45913f124a28 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:57.620907 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bc6dff-fb04-4c12-b0a3-45913f124a28 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:57.624699 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e7c723-8b40-408a-bfb1-1f6b7905b371 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:57.666189 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2592b72-dfc8-49c6-8953-5be9936342ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:57.666867 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2592b72-dfc8-49c6-8953-5be9936342ab None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:57.667150 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2592b72-dfc8-49c6-8953-5be9936342ab None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:57.667424 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2592b72-dfc8-49c6-8953-5be9936342ab None admin] PATH_INFO: `/v3/domains` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:57.668090 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2592b72-dfc8-49c6-8953-5be9936342ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:46:57.668397 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2592b72-dfc8-49c6-8953-5be9936342ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:57.669819 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2592b72-dfc8-49c6-8953-5be9936342ab None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:57.673842 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd10d2c-cb13-4b70-a769-9cfe6990d45f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:57.674920 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd10d2c-cb13-4b70-a769-9cfe6990d45f None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:57.675428 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd10d2c-cb13-4b70-a769-9cfe6990d45f None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:57.675897 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd10d2c-cb13-4b70-a769-9cfe6990d45f None admin] PATH_INFO: `/v3/projects` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:57.676801 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd10d2c-cb13-4b70-a769-9cfe6990d45f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:46:57.677570 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd10d2c-cb13-4b70-a769-9cfe6990d45f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:57.679682 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd10d2c-cb13-4b70-a769-9cfe6990d45f None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:57.686655 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 204/412] 10.222.0.59 () {56 vars in 1109 bytes} [Tue Oct 11 11:46:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:46:57.687329 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2592b72-dfc8-49c6-8953-5be9936342ab None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:57.701704 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 209/413] 10.222.0.59 () {56 vars in 1148 bytes} [Tue Oct 11 11:46:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:46:57.702603 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd10d2c-cb13-4b70-a769-9cfe6990d45f None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:57.707276 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:57.731876 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:57.886502 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd10d2c-cb13-4b70-a769-9cfe6990d45f None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:57.930899 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2592b72-dfc8-49c6-8953-5be9936342ab None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:57.962421 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968f8fd6-f710-457e-a55b-1b6859ddee2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:57.963300 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968f8fd6-f710-457e-a55b-1b6859ddee2a None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:57.967834 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968f8fd6-f710-457e-a55b-1b6859ddee2a None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:57.967834 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968f8fd6-f710-457e-a55b-1b6859ddee2a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:57.981525 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968f8fd6-f710-457e-a55b-1b6859ddee2a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:57.984111 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968f8fd6-f710-457e-a55b-1b6859ddee2a None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:58.008356 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application [None req-968f8fd6-f710-457e-a55b-1b6859ddee2a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 11:46:58.008356 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:46:58.008356 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:46:58.008356 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:46:58.008356 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:46:58.008356 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:46:58.008356 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:46:58.008356 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:46:58.008356 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:46:58.008356 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:46:58.008356 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:46:58.008356 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:46:58.008356 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 11 11:46:58.008356 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 11 11:46:58.008356 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 11 11:46:58.010034 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 11 11:46:58.010034 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:46:58.010034 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:46:58.010034 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:46:58.010034 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:46:58.010034 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:46:58.010034 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:46:58.010034 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:46:58.010034 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application with Lock( Oct 11 11:46:58.010034 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:46:58.010034 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:46:58.010034 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:46:58.010034 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:46:58.010034 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:46:58.010034 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:46:58.010034 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:46:58.010034 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:46:58.012471 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 11 11:46:58.012471 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 11 11:46:58.012471 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 11 11:46:58.012471 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 11 11:46:58.012471 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 11 11:46:58.012471 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 11 11:46:58.012471 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 11:46:58.012471 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:46:58.012471 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 210/414] 10.222.0.59 () {56 vars in 1111 bytes} [Tue Oct 11 11:46:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:46:58.012471 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968f8fd6-f710-457e-a55b-1b6859ddee2a None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:58.024558 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9616d154-fa4d-491c-9dab-0bb1629b3444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:58.024558 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9616d154-fa4d-491c-9dab-0bb1629b3444 None admin] REQUEST_METHOD: `PUT` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:58.024558 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9616d154-fa4d-491c-9dab-0bb1629b3444 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:58.024558 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9616d154-fa4d-491c-9dab-0bb1629b3444 None admin] PATH_INFO: `/v3/projects/26ecf552cc9a4e5ca1d380509683dc4a/users/2e563fe174db4cc19ae6bf8162cc3d23/roles/f5a43d8f3fa24a87a7abe7492cf6f87f` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:58.051538 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9616d154-fa4d-491c-9dab-0bb1629b3444 None admin] RBAC: Authorizing `identity:create_grant(project_id=26ecf552cc9a4e5ca1d380509683dc4a, user_id=2e563fe174db4cc19ae6bf8162cc3d23, role_id=f5a43d8f3fa24a87a7abe7492cf6f87f)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:58.054975 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9616d154-fa4d-491c-9dab-0bb1629b3444 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:58.083514 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 205/415] 10.222.0.59 () {58 vars in 1439 bytes} [Tue Oct 11 11:46:57 2022] PUT /identity/v3/projects/26ecf552cc9a4e5ca1d380509683dc4a/users/2e563fe174db4cc19ae6bf8162cc3d23/roles/f5a43d8f3fa24a87a7abe7492cf6f87f => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 11:46:58.086446 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9616d154-fa4d-491c-9dab-0bb1629b3444 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:58.155734 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:58.243605 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968f8fd6-f710-457e-a55b-1b6859ddee2a None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:58.246053 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:58.365602 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc464337-aaf7-401e-a2ac-9a6178687494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:58.369123 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc464337-aaf7-401e-a2ac-9a6178687494 None admin] REQUEST_METHOD: `PUT` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:58.370632 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc464337-aaf7-401e-a2ac-9a6178687494 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:58.371960 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc464337-aaf7-401e-a2ac-9a6178687494 None admin] PATH_INFO: `/v3/projects/26ecf552cc9a4e5ca1d380509683dc4a/users/bad5e1cb306b46818096940acc3810b2/roles/f5a43d8f3fa24a87a7abe7492cf6f87f` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:58.378487 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9616d154-fa4d-491c-9dab-0bb1629b3444 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:58.415001 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc464337-aaf7-401e-a2ac-9a6178687494 None admin] RBAC: Authorizing `identity:create_grant(project_id=26ecf552cc9a4e5ca1d380509683dc4a, user_id=bad5e1cb306b46818096940acc3810b2, role_id=f5a43d8f3fa24a87a7abe7492cf6f87f)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:58.420534 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc464337-aaf7-401e-a2ac-9a6178687494 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:58.448745 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 211/416] 10.222.0.59 () {58 vars in 1439 bytes} [Tue Oct 11 11:46:58 2022] PUT /identity/v3/projects/26ecf552cc9a4e5ca1d380509683dc4a/users/bad5e1cb306b46818096940acc3810b2/roles/f5a43d8f3fa24a87a7abe7492cf6f87f => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 11:46:58.480985 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7310c792-6b68-4678-af84-bb5d95f31139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:58.482474 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7310c792-6b68-4678-af84-bb5d95f31139 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:58.483214 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7310c792-6b68-4678-af84-bb5d95f31139 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:58.484090 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7310c792-6b68-4678-af84-bb5d95f31139 None admin] PATH_INFO: `/v3/projects` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:58.485620 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7310c792-6b68-4678-af84-bb5d95f31139 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:46:58.486374 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7310c792-6b68-4678-af84-bb5d95f31139 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:58.489705 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7310c792-6b68-4678-af84-bb5d95f31139 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:58.516943 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 206/417] 10.222.0.59 () {56 vars in 1148 bytes} [Tue Oct 11 11:46:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 431 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:46:58.523575 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc464337-aaf7-401e-a2ac-9a6178687494 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:58.755174 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb107e7-f4a2-4a3b-916b-6e5103d122b5 None None] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:58.756049 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb107e7-f4a2-4a3b-916b-6e5103d122b5 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:58.756049 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb107e7-f4a2-4a3b-916b-6e5103d122b5 None None] PATH_INFO: `/` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:58.756534 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc464337-aaf7-401e-a2ac-9a6178687494 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:58.757364 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 207/418] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:46:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:46:58.763582 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1ae24d-e6d7-4fe8-b59f-f137f6e07726 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:58.763845 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1ae24d-e6d7-4fe8-b59f-f137f6e07726 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:58.764130 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1ae24d-e6d7-4fe8-b59f-f137f6e07726 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:58.785704 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-7a1ae24d-e6d7-4fe8-b59f-f137f6e07726 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:46:58.787082 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-7a1ae24d-e6d7-4fe8-b59f-f137f6e07726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:46:58.795215 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:58.849102 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b8df20-d871-424d-bf1d-91b185770f84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:58.849102 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b8df20-d871-424d-bf1d-91b185770f84 None admin] REQUEST_METHOD: `PUT` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:58.849102 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b8df20-d871-424d-bf1d-91b185770f84 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:58.849102 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b8df20-d871-424d-bf1d-91b185770f84 None admin] PATH_INFO: `/v3/projects/26ecf552cc9a4e5ca1d380509683dc4a/users/2b1b651c73664d08accc6edf2f79bace/roles/f5a43d8f3fa24a87a7abe7492cf6f87f` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:58.879282 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b8df20-d871-424d-bf1d-91b185770f84 None admin] RBAC: Authorizing `identity:create_grant(project_id=26ecf552cc9a4e5ca1d380509683dc4a, user_id=2b1b651c73664d08accc6edf2f79bace, role_id=f5a43d8f3fa24a87a7abe7492cf6f87f)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:58.881752 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b8df20-d871-424d-bf1d-91b185770f84 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:58.903027 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 212/419] 10.222.0.59 () {58 vars in 1439 bytes} [Tue Oct 11 11:46:58 2022] PUT /identity/v3/projects/26ecf552cc9a4e5ca1d380509683dc4a/users/2b1b651c73664d08accc6edf2f79bace/roles/f5a43d8f3fa24a87a7abe7492cf6f87f => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 11:46:58.945485 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 208/420] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:46:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 11:46:59.387099 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b8df20-d871-424d-bf1d-91b185770f84 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:59.525798 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b8df20-d871-424d-bf1d-91b185770f84 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:59.575296 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fd13a7-c76a-4e57-88fb-d51c2193e3d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:59.575964 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fd13a7-c76a-4e57-88fb-d51c2193e3d3 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:59.576252 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fd13a7-c76a-4e57-88fb-d51c2193e3d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:59.576518 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fd13a7-c76a-4e57-88fb-d51c2193e3d3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:59.599893 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fd13a7-c76a-4e57-88fb-d51c2193e3d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 11:46:59.605044 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application [None req-12fd13a7-c76a-4e57-88fb-d51c2193e3d3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 11 11:46:59.605044 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:46:59.605044 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:46:59.605044 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:46:59.605044 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:46:59.605044 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:46:59.605044 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:46:59.605044 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:46:59.605044 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:46:59.605044 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:46:59.605044 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:46:59.605044 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:46:59.605044 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 11:46:59.605044 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 11:46:59.605044 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 11:46:59.605044 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 11:46:59.605974 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 11:46:59.605974 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 11:46:59.605974 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 11:46:59.605974 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 11:46:59.605974 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:46:59.605974 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:46:59.605974 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:46:59.605974 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:46:59.605974 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:46:59.605974 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:46:59.605974 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:46:59.605974 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application with Lock( Oct 11 11:46:59.605974 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:46:59.605974 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:46:59.605974 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:46:59.605974 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:46:59.607227 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:46:59.607227 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:46:59.607227 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:46:59.607227 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:46:59.607227 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 11:46:59.607227 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 11:46:59.607227 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 11:46:59.607227 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 11:46:59.607227 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 11:46:59.607227 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 11:46:59.607227 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 11 11:46:59.607227 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:46:59.607227 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 213/421] 10.222.0.59 () {56 vars in 1102 bytes} [Tue Oct 11 11:46:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:46:59.614462 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1ae24d-e6d7-4fe8-b59f-f137f6e07726 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:59.685291 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:46:59.757999 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1ae24d-e6d7-4fe8-b59f-f137f6e07726 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:46:59.825110 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddfa7b6-106a-48e4-b75e-9f605350c390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:46:59.825750 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddfa7b6-106a-48e4-b75e-9f605350c390 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:46:59.826033 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddfa7b6-106a-48e4-b75e-9f605350c390 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:46:59.826307 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddfa7b6-106a-48e4-b75e-9f605350c390 None admin] PATH_INFO: `/v3/roles` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:46:59.827025 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddfa7b6-106a-48e4-b75e-9f605350c390 None admin] RBAC: Adding query filter params (name=service) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:46:59.827335 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddfa7b6-106a-48e4-b75e-9f605350c390 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:46:59.828755 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddfa7b6-106a-48e4-b75e-9f605350c390 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:46:59.841693 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 209/422] 10.222.0.59 () {56 vars in 1103 bytes} [Tue Oct 11 11:46:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:46:59.848616 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fd13a7-c76a-4e57-88fb-d51c2193e3d3 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:46:59.918989 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:00.016126 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806ff04d-b2b5-4fdb-928c-842ac77fc5a8 None None] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:00.016629 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806ff04d-b2b5-4fdb-928c-842ac77fc5a8 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:00.017105 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806ff04d-b2b5-4fdb-928c-842ac77fc5a8 None None] PATH_INFO: `/` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:00.021310 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 210/423] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:47:00 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 11 11:47:00.029394 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b004b45-d03a-404c-af80-3b3271d7a627 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:00.029691 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b004b45-d03a-404c-af80-3b3271d7a627 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:00.029959 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b004b45-d03a-404c-af80-3b3271d7a627 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:00.049641 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fd13a7-c76a-4e57-88fb-d51c2193e3d3 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:00.071255 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-4b004b45-d03a-404c-af80-3b3271d7a627 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:47:00.072491 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-4b004b45-d03a-404c-af80-3b3271d7a627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:47:00.135347 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f55c9ab-4693-4c29-a59a-4b1fa4f00f40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:00.136009 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f55c9ab-4693-4c29-a59a-4b1fa4f00f40 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:00.136289 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f55c9ab-4693-4c29-a59a-4b1fa4f00f40 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:00.136562 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f55c9ab-4693-4c29-a59a-4b1fa4f00f40 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:00.156900 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f55c9ab-4693-4c29-a59a-4b1fa4f00f40 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:00.158478 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f55c9ab-4693-4c29-a59a-4b1fa4f00f40 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:00.181131 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application [None req-5f55c9ab-4693-4c29-a59a-4b1fa4f00f40 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 11:47:00.181131 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:47:00.181131 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:47:00.181131 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:47:00.181131 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:47:00.181131 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:47:00.181131 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:47:00.181131 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:47:00.181131 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:47:00.181131 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:47:00.181131 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:47:00.181131 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:47:00.181131 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 11:47:00.181131 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 11:47:00.181131 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 11:47:00.181131 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 11:47:00.186866 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:47:00.186866 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:47:00.186866 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:47:00.186866 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:47:00.186866 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:47:00.186866 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:47:00.186866 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:47:00.186866 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application with Lock( Oct 11 11:47:00.186866 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:47:00.186866 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:47:00.186866 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:47:00.186866 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:47:00.186866 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:47:00.186866 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:47:00.186866 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:47:00.186866 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:47:00.186866 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 11:47:00.188806 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 11:47:00.188806 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 11:47:00.188806 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:47:00.188806 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 214/424] 10.222.0.59 () {56 vars in 1108 bytes} [Tue Oct 11 11:46:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 11 11:47:00.188806 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f55c9ab-4693-4c29-a59a-4b1fa4f00f40 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:00.207989 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 211/425] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:47:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 11:47:00.274460 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:00.303061 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f55c9ab-4693-4c29-a59a-4b1fa4f00f40 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:00.340794 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248d7df6-43fe-49ea-a182-c093523d1178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:00.341442 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248d7df6-43fe-49ea-a182-c093523d1178 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:00.341722 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248d7df6-43fe-49ea-a182-c093523d1178 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:00.342001 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248d7df6-43fe-49ea-a182-c093523d1178 None admin] PATH_INFO: `/v3/domains` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:00.342707 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248d7df6-43fe-49ea-a182-c093523d1178 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:00.343014 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248d7df6-43fe-49ea-a182-c093523d1178 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:00.344455 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248d7df6-43fe-49ea-a182-c093523d1178 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:00.359092 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 215/426] 10.222.0.59 () {56 vars in 1109 bytes} [Tue Oct 11 11:47:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 11:47:00.367892 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b004b45-d03a-404c-af80-3b3271d7a627 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:00.494364 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:00.510084 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248d7df6-43fe-49ea-a182-c093523d1178 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:00.590519 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b004b45-d03a-404c-af80-3b3271d7a627 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:00.653224 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee20ac74-5dd5-411f-80cb-d1e5eea09321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:00.654304 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee20ac74-5dd5-411f-80cb-d1e5eea09321 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:00.654661 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee20ac74-5dd5-411f-80cb-d1e5eea09321 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:00.655141 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee20ac74-5dd5-411f-80cb-d1e5eea09321 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:00.669636 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee20ac74-5dd5-411f-80cb-d1e5eea09321 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:00.672428 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee20ac74-5dd5-411f-80cb-d1e5eea09321 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:00.696264 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application [None req-ee20ac74-5dd5-411f-80cb-d1e5eea09321 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 11 11:47:00.696264 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:47:00.696264 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:47:00.696264 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:47:00.696264 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:47:00.696264 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:47:00.696264 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:47:00.696264 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:47:00.696264 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:47:00.696264 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:47:00.696264 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:47:00.696264 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:47:00.696264 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 11 11:47:00.696264 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 11 11:47:00.696264 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 11 11:47:00.696264 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 11 11:47:00.697244 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:47:00.697244 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:47:00.697244 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 11 11:47:00.697244 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 11:47:00.697244 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 11 11:47:00.697244 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 11:47:00.697244 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:47:00.697244 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:47:00.697244 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:47:00.697244 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:47:00.697244 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:47:00.697244 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application with Lock( Oct 11 11:47:00.697244 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:47:00.697244 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:47:00.697244 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:47:00.697244 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:47:00.697244 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:47:00.697244 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:47:00.698109 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:47:00.698109 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:47:00.698109 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 11 11:47:00.698109 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 11 11:47:00.698109 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 11 11:47:00.698109 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 11 11:47:00.698109 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 11 11:47:00.698109 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 11 11:47:00.698109 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 11 11:47:00.698109 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:47:00.699119 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 212/427] 10.222.0.59 () {56 vars in 1099 bytes} [Tue Oct 11 11:47:00 2022] GET /identity/v3/users/glance => generated 84 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 11 11:47:00.707311 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee20ac74-5dd5-411f-80cb-d1e5eea09321 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:00.799038 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248d7df6-43fe-49ea-a182-c093523d1178 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:00.841791 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee20ac74-5dd5-411f-80cb-d1e5eea09321 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:00.853451 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:00.884098 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069652a4-7ab7-44e5-89b8-7e5bec5ab87a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:00.885929 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069652a4-7ab7-44e5-89b8-7e5bec5ab87a None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:00.886767 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069652a4-7ab7-44e5-89b8-7e5bec5ab87a None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:00.887749 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069652a4-7ab7-44e5-89b8-7e5bec5ab87a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:00.888609 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e080e7-f987-4adb-9257-23643aced21d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:00.889983 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e080e7-f987-4adb-9257-23643aced21d None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:00.891799 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e080e7-f987-4adb-9257-23643aced21d None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:00.892334 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e080e7-f987-4adb-9257-23643aced21d None admin] PATH_INFO: `/v3/users` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:00.893391 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e080e7-f987-4adb-9257-23643aced21d None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:00.894226 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e080e7-f987-4adb-9257-23643aced21d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:00.895911 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:00.897083 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e080e7-f987-4adb-9257-23643aced21d None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:00.916600 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069652a4-7ab7-44e5-89b8-7e5bec5ab87a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 11:47:00.928991 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application [None req-069652a4-7ab7-44e5-89b8-7e5bec5ab87a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 11 11:47:00.928991 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:47:00.928991 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:47:00.928991 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:47:00.928991 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:47:00.928991 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:47:00.928991 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:47:00.928991 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:47:00.928991 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:47:00.928991 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:47:00.928991 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:47:00.928991 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:47:00.928991 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 11:47:00.928991 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 11:47:00.928991 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 11:47:00.928991 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 11:47:00.930864 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 11:47:00.930864 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 11:47:00.930864 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 11:47:00.930864 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 11:47:00.930864 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:47:00.930864 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:47:00.930864 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:47:00.930864 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:47:00.930864 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:47:00.930864 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:47:00.930864 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:47:00.930864 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application with Lock( Oct 11 11:47:00.930864 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:47:00.930864 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:47:00.930864 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:47:00.930864 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:47:00.932370 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:47:00.932370 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:47:00.932370 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:47:00.932370 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:47:00.932370 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 11:47:00.932370 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 11:47:00.932370 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 11:47:00.932370 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 11:47:00.932370 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 11:47:00.932370 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 11:47:00.932370 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 11 11:47:00.932370 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:47:00.932370 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 216/428] 10.222.0.59 () {56 vars in 1102 bytes} [Tue Oct 11 11:47:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 421 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:47:00.935886 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 213/429] 10.222.0.59 () {56 vars in 1137 bytes} [Tue Oct 11 11:47:00 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:47:00.940960 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069652a4-7ab7-44e5-89b8-7e5bec5ab87a None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:00.945584 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e080e7-f987-4adb-9257-23643aced21d None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:01.127234 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069652a4-7ab7-44e5-89b8-7e5bec5ab87a None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:01.147467 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e080e7-f987-4adb-9257-23643aced21d None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:01.194978 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f412fa9f-6b05-4e10-99f5-a61d316c2c4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:01.196023 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f412fa9f-6b05-4e10-99f5-a61d316c2c4a None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:01.196570 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f412fa9f-6b05-4e10-99f5-a61d316c2c4a None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:01.197030 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f412fa9f-6b05-4e10-99f5-a61d316c2c4a None admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:01.197892 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f412fa9f-6b05-4e10-99f5-a61d316c2c4a None admin] RBAC: Adding query filter params (name=service) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:01.198416 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f412fa9f-6b05-4e10-99f5-a61d316c2c4a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:01.200057 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f412fa9f-6b05-4e10-99f5-a61d316c2c4a None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:01.214954 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 217/430] 10.222.0.59 () {56 vars in 1103 bytes} [Tue Oct 11 11:47:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:47:01.219635 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd11fd5a-4c0d-4762-ada8-ec15db8e3812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:01.220409 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd11fd5a-4c0d-4762-ada8-ec15db8e3812 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:01.220963 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd11fd5a-4c0d-4762-ada8-ec15db8e3812 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:01.221441 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd11fd5a-4c0d-4762-ada8-ec15db8e3812 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:01.221733 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f412fa9f-6b05-4e10-99f5-a61d316c2c4a None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:01.298505 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:01.340697 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:01.368369 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd11fd5a-4c0d-4762-ada8-ec15db8e3812 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:01.369871 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd11fd5a-4c0d-4762-ada8-ec15db8e3812 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:01.431804 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f412fa9f-6b05-4e10-99f5-a61d316c2c4a None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:01.482368 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f99204-4559-4624-a718-c430df68f717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:01.484242 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f99204-4559-4624-a718-c430df68f717 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:01.485038 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f99204-4559-4624-a718-c430df68f717 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:01.485577 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f99204-4559-4624-a718-c430df68f717 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:01.500221 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f99204-4559-4624-a718-c430df68f717 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:01.501743 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f99204-4559-4624-a718-c430df68f717 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:01.517924 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application [None req-33f99204-4559-4624-a718-c430df68f717 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 11:47:01.517924 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:47:01.517924 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:47:01.517924 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:47:01.517924 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:47:01.517924 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:47:01.517924 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:47:01.517924 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:47:01.517924 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:47:01.517924 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:47:01.517924 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:47:01.517924 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:47:01.517924 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 11:47:01.517924 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 11:47:01.517924 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 11:47:01.517924 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 11:47:01.518946 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:47:01.518946 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:47:01.518946 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:47:01.518946 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:47:01.518946 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:47:01.518946 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:47:01.518946 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:47:01.518946 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application with Lock( Oct 11 11:47:01.518946 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:47:01.518946 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:47:01.518946 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:47:01.518946 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:47:01.518946 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:47:01.518946 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:47:01.518946 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:47:01.518946 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:47:01.518946 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 11:47:01.519791 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 11:47:01.519791 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 11:47:01.519791 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:47:01.519791 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 218/431] 10.222.0.59 () {56 vars in 1108 bytes} [Tue Oct 11 11:47:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:47:01.525643 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d70ffa-1793-41ae-a3ac-abeddddf7bd9 None None] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:01.525939 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d70ffa-1793-41ae-a3ac-abeddddf7bd9 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:01.526197 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d70ffa-1793-41ae-a3ac-abeddddf7bd9 None None] PATH_INFO: `/` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:01.531996 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 219/432] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:47:01 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:47:01.531996 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d70ffa-1793-41ae-a3ac-abeddddf7bd9 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:01.539797 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 214/433] 10.222.0.59 () {58 vars in 1282 bytes} [Tue Oct 11 11:47:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 595 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 11 11:47:01.541839 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d4ede0-73cc-4e30-b7cd-25a7ffebeac6 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:01.542450 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d4ede0-73cc-4e30-b7cd-25a7ffebeac6 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:01.543021 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d4ede0-73cc-4e30-b7cd-25a7ffebeac6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:01.566325 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-a6d4ede0-73cc-4e30-b7cd-25a7ffebeac6 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:47:01.568250 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-a6d4ede0-73cc-4e30-b7cd-25a7ffebeac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:47:01.646968 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d70ffa-1793-41ae-a3ac-abeddddf7bd9 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:01.651720 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:01.660741 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:01.664962 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 215/434] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:47:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 11:47:01.667573 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d4ede0-73cc-4e30-b7cd-25a7ffebeac6 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:01.714781 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69659fe1-e439-4a58-801a-462cea1313d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:01.715649 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69659fe1-e439-4a58-801a-462cea1313d0 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:01.717462 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69659fe1-e439-4a58-801a-462cea1313d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:01.717934 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69659fe1-e439-4a58-801a-462cea1313d0 None admin] PATH_INFO: `/v3/domains` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:01.719088 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69659fe1-e439-4a58-801a-462cea1313d0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:01.720642 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69659fe1-e439-4a58-801a-462cea1313d0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:01.722595 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69659fe1-e439-4a58-801a-462cea1313d0 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:01.752420 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 220/435] 10.222.0.59 () {56 vars in 1109 bytes} [Tue Oct 11 11:47:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:47:01.768304 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69659fe1-e439-4a58-801a-462cea1313d0 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:01.874345 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d4ede0-73cc-4e30-b7cd-25a7ffebeac6 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:01.905016 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69659fe1-e439-4a58-801a-462cea1313d0 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:01.938808 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8f8c15-eacf-4877-aa54-b3ee49751cba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:01.938808 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8f8c15-eacf-4877-aa54-b3ee49751cba None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:01.938808 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8f8c15-eacf-4877-aa54-b3ee49751cba None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:01.938808 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8f8c15-eacf-4877-aa54-b3ee49751cba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:01.952260 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8f8c15-eacf-4877-aa54-b3ee49751cba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:01.958517 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8f8c15-eacf-4877-aa54-b3ee49751cba None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:01.974144 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7198829e-10b0-4814-83da-2993629fe6e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:01.975139 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7198829e-10b0-4814-83da-2993629fe6e7 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:01.976334 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7198829e-10b0-4814-83da-2993629fe6e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:01.976620 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7198829e-10b0-4814-83da-2993629fe6e7 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:01.985630 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7198829e-10b0-4814-83da-2993629fe6e7 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:01.987282 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7198829e-10b0-4814-83da-2993629fe6e7 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:01.991586 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application [None req-3c8f8c15-eacf-4877-aa54-b3ee49751cba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 11:47:01.991586 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:47:01.991586 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:47:01.991586 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:47:01.991586 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:47:01.991586 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:47:01.991586 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:47:01.991586 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:47:01.991586 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:47:01.991586 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:47:01.991586 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:47:01.991586 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:47:01.991586 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 11:47:01.991586 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 11:47:01.991586 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 11:47:01.991586 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 11:47:01.992512 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:47:01.992512 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:47:01.992512 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:47:01.992512 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:47:01.992512 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:47:01.992512 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:47:01.992512 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:47:01.992512 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application with Lock( Oct 11 11:47:01.992512 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:47:01.992512 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:47:01.992512 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:47:01.992512 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:47:01.992512 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:47:01.992512 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:47:01.992512 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:47:01.992512 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:47:01.992512 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 11:47:01.993376 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 11:47:01.993376 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 11:47:01.993376 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:47:01.994056 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 216/436] 10.222.0.59 () {56 vars in 1108 bytes} [Tue Oct 11 11:47:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:47:01.998434 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99accfe-eef1-422d-89c1-bcd70b9634e8 None None] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:01.998434 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99accfe-eef1-422d-89c1-bcd70b9634e8 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:01.998434 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99accfe-eef1-422d-89c1-bcd70b9634e8 None None] PATH_INFO: `/` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:01.999836 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 217/437] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:47:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:47:02.001120 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application [None req-7198829e-10b0-4814-83da-2993629fe6e7 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 11 11:47:02.001120 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:47:02.001120 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:47:02.001120 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:47:02.001120 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:47:02.001120 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:47:02.001120 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:47:02.001120 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:47:02.001120 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:47:02.001120 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:47:02.001120 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:47:02.001120 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:47:02.001120 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 11 11:47:02.001120 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 11 11:47:02.001120 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 11 11:47:02.001120 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 11 11:47:02.002077 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:47:02.002077 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:47:02.002077 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 11 11:47:02.002077 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 11:47:02.002077 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 11 11:47:02.002077 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 11:47:02.002077 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:47:02.002077 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:47:02.002077 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:47:02.002077 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:47:02.002077 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:47:02.002077 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application with Lock( Oct 11 11:47:02.002077 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:47:02.002077 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:47:02.002077 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:47:02.002077 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:47:02.002077 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:47:02.002077 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:47:02.003024 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:47:02.003024 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:47:02.003024 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 11 11:47:02.003024 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 11 11:47:02.003024 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 11 11:47:02.003024 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 11 11:47:02.003024 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 11 11:47:02.003024 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 11 11:47:02.003024 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 11 11:47:02.003024 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:47:02.005676 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb1b94a-2e92-49ce-aec6-bc1d376d10fd None None] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:02.006140 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb1b94a-2e92-49ce-aec6-bc1d376d10fd None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:02.006951 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 221/438] 10.222.0.59 () {56 vars in 1096 bytes} [Tue Oct 11 11:47:01 2022] GET /identity/v3/users/swift => generated 83 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 11 11:47:02.008168 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7198829e-10b0-4814-83da-2993629fe6e7 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:02.010100 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb1b94a-2e92-49ce-aec6-bc1d376d10fd None None] PATH_INFO: `/` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:02.012961 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 218/439] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:47:02 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:47:02.014544 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb1b94a-2e92-49ce-aec6-bc1d376d10fd None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:02.039510 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:02.079648 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:02.123663 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7198829e-10b0-4814-83da-2993629fe6e7 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:02.141828 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb1b94a-2e92-49ce-aec6-bc1d376d10fd None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:02.165364 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1392fdf2-1d53-4a8c-809f-a28dac3cf712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:02.166016 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1392fdf2-1d53-4a8c-809f-a28dac3cf712 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:02.166299 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1392fdf2-1d53-4a8c-809f-a28dac3cf712 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:02.166652 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1392fdf2-1d53-4a8c-809f-a28dac3cf712 None admin] PATH_INFO: `/v3/domains` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:02.167412 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1392fdf2-1d53-4a8c-809f-a28dac3cf712 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:02.167871 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1392fdf2-1d53-4a8c-809f-a28dac3cf712 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:02.169739 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1392fdf2-1d53-4a8c-809f-a28dac3cf712 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:02.180125 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35a5902-bae2-435e-9dff-2d85a6e2df61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:02.180748 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35a5902-bae2-435e-9dff-2d85a6e2df61 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:02.181027 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35a5902-bae2-435e-9dff-2d85a6e2df61 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:02.181299 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35a5902-bae2-435e-9dff-2d85a6e2df61 None admin] PATH_INFO: `/v3/users` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:02.182039 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35a5902-bae2-435e-9dff-2d85a6e2df61 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:02.182350 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35a5902-bae2-435e-9dff-2d85a6e2df61 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:02.183845 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35a5902-bae2-435e-9dff-2d85a6e2df61 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:02.184189 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 222/440] 10.222.0.59 () {56 vars in 1109 bytes} [Tue Oct 11 11:47:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:47:02.185464 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eada145-0356-4e08-8a5e-50e09ab44566 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:02.185751 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eada145-0356-4e08-8a5e-50e09ab44566 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:02.186025 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eada145-0356-4e08-8a5e-50e09ab44566 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:02.204451 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-5eada145-0356-4e08-8a5e-50e09ab44566 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:47:02.205755 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-5eada145-0356-4e08-8a5e-50e09ab44566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:47:02.207434 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 219/441] 10.222.0.59 () {56 vars in 1135 bytes} [Tue Oct 11 11:47:02 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:47:02.208321 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35a5902-bae2-435e-9dff-2d85a6e2df61 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:02.295147 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 223/442] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:47:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 11:47:02.296531 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b038f77-6a87-4b40-a117-c9d8b95fb10c None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:02.296883 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b038f77-6a87-4b40-a117-c9d8b95fb10c None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:02.297217 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b038f77-6a87-4b40-a117-c9d8b95fb10c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:02.315621 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-6b038f77-6a87-4b40-a117-c9d8b95fb10c None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:47:02.316921 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-6b038f77-6a87-4b40-a117-c9d8b95fb10c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:47:02.331184 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35a5902-bae2-435e-9dff-2d85a6e2df61 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:02.331803 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:02.343302 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:02.382847 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f5992e-f87d-44cf-8b02-c2ae58ceade6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:02.383909 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f5992e-f87d-44cf-8b02-c2ae58ceade6 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:02.384398 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f5992e-f87d-44cf-8b02-c2ae58ceade6 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:02.384863 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f5992e-f87d-44cf-8b02-c2ae58ceade6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:02.403986 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f5992e-f87d-44cf-8b02-c2ae58ceade6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 11:47:02.406771 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 224/443] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:47:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 11:47:02.407911 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b038f77-6a87-4b40-a117-c9d8b95fb10c None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:02.414281 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application [None req-26f5992e-f87d-44cf-8b02-c2ae58ceade6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 11 11:47:02.414281 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:47:02.414281 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:47:02.414281 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:47:02.414281 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:47:02.414281 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:47:02.414281 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:47:02.414281 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:47:02.414281 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:47:02.414281 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:47:02.414281 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:47:02.414281 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:47:02.414281 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 11:47:02.414281 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 11:47:02.414281 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 11:47:02.414281 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 11:47:02.415405 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 11:47:02.415405 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 11:47:02.415405 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 11:47:02.415405 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 11:47:02.415405 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:47:02.415405 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:47:02.415405 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:47:02.415405 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:47:02.415405 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:47:02.415405 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:47:02.415405 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:47:02.415405 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application with Lock( Oct 11 11:47:02.415405 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:47:02.415405 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:47:02.415405 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:47:02.415405 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:47:02.416302 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:47:02.416302 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:47:02.416302 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:47:02.416302 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:47:02.416302 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 11:47:02.416302 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 11:47:02.416302 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 11:47:02.416302 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 11:47:02.416302 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 11:47:02.416302 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 11:47:02.416302 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 11 11:47:02.416302 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:47:02.417111 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 220/444] 10.222.0.59 () {56 vars in 1102 bytes} [Tue Oct 11 11:47:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:47:02.418344 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f5992e-f87d-44cf-8b02-c2ae58ceade6 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:02.555629 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b038f77-6a87-4b40-a117-c9d8b95fb10c None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:02.596369 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f5992e-f87d-44cf-8b02-c2ae58ceade6 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:02.611945 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0cf788-f1de-48b4-bf96-6a58812cabbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:02.613003 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0cf788-f1de-48b4-bf96-6a58812cabbf None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:02.613553 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0cf788-f1de-48b4-bf96-6a58812cabbf None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:02.614040 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0cf788-f1de-48b4-bf96-6a58812cabbf None admin] PATH_INFO: `/v3/projects/service` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:02.626690 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0cf788-f1de-48b4-bf96-6a58812cabbf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:02.629447 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0cf788-f1de-48b4-bf96-6a58812cabbf None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:02.633731 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23eaffc-d114-4437-a428-9d0770c6163a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:02.634384 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23eaffc-d114-4437-a428-9d0770c6163a None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:02.634692 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23eaffc-d114-4437-a428-9d0770c6163a None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:02.635032 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23eaffc-d114-4437-a428-9d0770c6163a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:02.650104 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23eaffc-d114-4437-a428-9d0770c6163a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:02.651027 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application [None req-7f0cf788-f1de-48b4-bf96-6a58812cabbf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 11:47:02.651027 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:47:02.651027 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:47:02.651027 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:47:02.651027 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:47:02.651027 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:47:02.651027 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:47:02.651027 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:47:02.651027 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:47:02.651027 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:47:02.651027 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:47:02.651027 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:47:02.651027 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 11 11:47:02.651027 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 11 11:47:02.651027 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 11 11:47:02.651837 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 11 11:47:02.651837 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:47:02.651837 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:47:02.651837 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:47:02.651837 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:47:02.651837 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:47:02.651837 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:47:02.651837 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:47:02.651837 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application with Lock( Oct 11 11:47:02.651837 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:47:02.651837 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:47:02.651837 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:47:02.651837 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:47:02.651837 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:47:02.651837 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:47:02.651837 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:47:02.651837 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:47:02.652678 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 11 11:47:02.652678 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 11 11:47:02.652678 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 11 11:47:02.652678 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 11 11:47:02.652678 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 11 11:47:02.652678 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 11 11:47:02.652678 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 11:47:02.652678 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:47:02.653114 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23eaffc-d114-4437-a428-9d0770c6163a None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:02.660602 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 225/445] 10.222.0.59 () {56 vars in 1111 bytes} [Tue Oct 11 11:47:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:47:02.662051 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0cf788-f1de-48b4-bf96-6a58812cabbf None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:02.671219 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application [None req-b23eaffc-d114-4437-a428-9d0770c6163a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 11:47:02.671219 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:47:02.671219 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:47:02.671219 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:47:02.671219 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:47:02.671219 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:47:02.671219 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:47:02.671219 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:47:02.671219 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:47:02.671219 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:47:02.671219 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:47:02.671219 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:47:02.671219 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 11:47:02.671219 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 11:47:02.671219 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 11:47:02.671219 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 11:47:02.672421 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:47:02.672421 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:47:02.672421 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:47:02.672421 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:47:02.672421 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:47:02.672421 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:47:02.672421 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:47:02.672421 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application with Lock( Oct 11 11:47:02.672421 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:47:02.672421 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:47:02.672421 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:47:02.672421 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:47:02.672421 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:47:02.672421 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:47:02.672421 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:47:02.672421 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:47:02.672421 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 11:47:02.673520 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 11:47:02.673520 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 11:47:02.673520 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:47:02.673520 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 221/446] 10.222.0.59 () {56 vars in 1108 bytes} [Tue Oct 11 11:47:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:47:02.673520 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23eaffc-d114-4437-a428-9d0770c6163a None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:02.689208 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:02.697652 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:02.789465 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23eaffc-d114-4437-a428-9d0770c6163a None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:02.828490 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0cf788-f1de-48b4-bf96-6a58812cabbf None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:02.829468 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50dbb58-6284-426a-9d39-77bad2270bed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:02.830151 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50dbb58-6284-426a-9d39-77bad2270bed None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:02.830433 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50dbb58-6284-426a-9d39-77bad2270bed None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:02.830754 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50dbb58-6284-426a-9d39-77bad2270bed None admin] PATH_INFO: `/v3/roles/service` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:02.843550 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50dbb58-6284-426a-9d39-77bad2270bed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 11:47:02.849223 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application [None req-b50dbb58-6284-426a-9d39-77bad2270bed None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 11 11:47:02.849223 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:47:02.849223 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:47:02.849223 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:47:02.849223 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:47:02.849223 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:47:02.849223 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:47:02.849223 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:47:02.849223 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:47:02.849223 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:47:02.849223 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:47:02.849223 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:47:02.849223 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 11:47:02.849223 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 11:47:02.849223 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 11:47:02.849223 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 11:47:02.854823 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 11:47:02.854823 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 11:47:02.854823 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 11:47:02.854823 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 11:47:02.854823 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:47:02.854823 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:47:02.854823 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:47:02.854823 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:47:02.854823 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:47:02.854823 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:47:02.854823 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:47:02.854823 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application with Lock( Oct 11 11:47:02.854823 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:47:02.854823 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:47:02.854823 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:47:02.854823 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:47:02.855800 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:47:02.855800 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:47:02.855800 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:47:02.855800 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:47:02.855800 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 11:47:02.855800 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 11:47:02.855800 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 11:47:02.855800 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 11:47:02.855800 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 11:47:02.855800 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 11:47:02.855800 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 11 11:47:02.855800 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:47:02.855800 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 222/447] 10.222.0.59 () {56 vars in 1102 bytes} [Tue Oct 11 11:47:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:47:02.855800 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50dbb58-6284-426a-9d39-77bad2270bed None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:02.865967 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08f1ec7-e34a-470e-b546-162bf1dac951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:02.866730 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08f1ec7-e34a-470e-b546-162bf1dac951 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:02.867136 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08f1ec7-e34a-470e-b546-162bf1dac951 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:02.867532 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08f1ec7-e34a-470e-b546-162bf1dac951 None admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:02.868300 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08f1ec7-e34a-470e-b546-162bf1dac951 None admin] RBAC: Adding query filter params (name=service) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:02.868685 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08f1ec7-e34a-470e-b546-162bf1dac951 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:02.870232 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08f1ec7-e34a-470e-b546-162bf1dac951 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:02.879067 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 226/448] 10.222.0.59 () {56 vars in 1103 bytes} [Tue Oct 11 11:47:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:47:02.879929 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08f1ec7-e34a-470e-b546-162bf1dac951 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:02.946411 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:02.966523 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:02.985190 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50dbb58-6284-426a-9d39-77bad2270bed None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:03.001337 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08f1ec7-e34a-470e-b546-162bf1dac951 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:03.024928 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec75697-e56e-46ba-8a8c-a47afbacb340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:03.025553 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec75697-e56e-46ba-8a8c-a47afbacb340 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:03.025834 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec75697-e56e-46ba-8a8c-a47afbacb340 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:03.026106 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec75697-e56e-46ba-8a8c-a47afbacb340 None admin] PATH_INFO: `/v3/projects` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:03.026836 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec75697-e56e-46ba-8a8c-a47afbacb340 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:03.027140 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec75697-e56e-46ba-8a8c-a47afbacb340 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:03.028606 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec75697-e56e-46ba-8a8c-a47afbacb340 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:03.040358 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c05468-471b-4f3a-b283-c1938414afda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:03.040989 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c05468-471b-4f3a-b283-c1938414afda None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:03.041952 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c05468-471b-4f3a-b283-c1938414afda None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:03.041952 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c05468-471b-4f3a-b283-c1938414afda None admin] PATH_INFO: `/v3/domains` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:03.042144 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 223/449] 10.222.0.59 () {56 vars in 1148 bytes} [Tue Oct 11 11:47:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:47:03.042580 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec75697-e56e-46ba-8a8c-a47afbacb340 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:03.043138 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c05468-471b-4f3a-b283-c1938414afda None admin] RBAC: Adding query filter params (name=Default) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:03.043459 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c05468-471b-4f3a-b283-c1938414afda None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:03.044905 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c05468-471b-4f3a-b283-c1938414afda None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:03.061477 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 227/450] 10.222.0.59 () {56 vars in 1109 bytes} [Tue Oct 11 11:47:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:47:03.062369 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c05468-471b-4f3a-b283-c1938414afda None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:03.162871 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec75697-e56e-46ba-8a8c-a47afbacb340 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:03.170189 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c05468-471b-4f3a-b283-c1938414afda None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:03.202422 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:03.205317 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9a3d8b-df1c-4ba1-9283-0758a2e87dc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:03.205984 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9a3d8b-df1c-4ba1-9283-0758a2e87dc6 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:03.206262 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9a3d8b-df1c-4ba1-9283-0758a2e87dc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:03.206540 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9a3d8b-df1c-4ba1-9283-0758a2e87dc6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:03.210215 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c44d86-1d23-4f34-99c1-bd69baa2ad4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:03.210784 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c44d86-1d23-4f34-99c1-bd69baa2ad4a None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:03.211061 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c44d86-1d23-4f34-99c1-bd69baa2ad4a None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:03.211332 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c44d86-1d23-4f34-99c1-bd69baa2ad4a None admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:03.212002 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c44d86-1d23-4f34-99c1-bd69baa2ad4a None admin] RBAC: Adding query filter params (name=service) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:03.212306 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c44d86-1d23-4f34-99c1-bd69baa2ad4a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:03.213720 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c44d86-1d23-4f34-99c1-bd69baa2ad4a None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:03.214238 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:03.218668 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9a3d8b-df1c-4ba1-9283-0758a2e87dc6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 11:47:03.228199 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 228/451] 10.222.0.59 () {56 vars in 1103 bytes} [Tue Oct 11 11:47:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:47:03.228199 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c44d86-1d23-4f34-99c1-bd69baa2ad4a None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:03.228478 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application [None req-da9a3d8b-df1c-4ba1-9283-0758a2e87dc6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 11 11:47:03.228478 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:47:03.228478 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:47:03.228478 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:47:03.228478 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:47:03.228478 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:47:03.228478 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:47:03.228478 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:47:03.228478 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:47:03.228478 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:47:03.228478 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:47:03.228478 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:47:03.228478 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 11:47:03.228478 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 11:47:03.228478 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 11:47:03.228478 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 11:47:03.229289 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 11:47:03.229289 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 11:47:03.229289 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 11:47:03.229289 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 11:47:03.229289 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:47:03.229289 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:47:03.229289 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:47:03.229289 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:47:03.229289 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:47:03.229289 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:47:03.229289 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:47:03.229289 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application with Lock( Oct 11 11:47:03.229289 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:47:03.229289 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:47:03.229289 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:47:03.229289 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:47:03.230067 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:47:03.230067 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:47:03.230067 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:47:03.230067 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:47:03.230067 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 11:47:03.230067 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 11:47:03.230067 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 11:47:03.230067 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 11:47:03.230067 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 11:47:03.230067 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 11:47:03.230067 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 11 11:47:03.230067 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:47:03.230067 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 224/452] 10.222.0.59 () {56 vars in 1102 bytes} [Tue Oct 11 11:47:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:47:03.230067 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9a3d8b-df1c-4ba1-9283-0758a2e87dc6 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:03.337596 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9a3d8b-df1c-4ba1-9283-0758a2e87dc6 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:03.339406 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c44d86-1d23-4f34-99c1-bd69baa2ad4a None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:03.375872 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e773f0-7fd1-44a7-9700-b6ed93021f2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:03.376641 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e773f0-7fd1-44a7-9700-b6ed93021f2e None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:03.376922 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e773f0-7fd1-44a7-9700-b6ed93021f2e None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:03.377197 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e773f0-7fd1-44a7-9700-b6ed93021f2e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:03.378073 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e8d78d-3bc9-44ac-b061-4f138ded2c78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:03.378245 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e773f0-7fd1-44a7-9700-b6ed93021f2e None admin] RBAC: Adding query filter params (role.id=f5a43d8f3fa24a87a7abe7492cf6f87f, scope.project.id=26ecf552cc9a4e5ca1d380509683dc4a, user.id=91a00658b79a42bd91a22a2508b5de34) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:03.378395 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e773f0-7fd1-44a7-9700-b6ed93021f2e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:03.378745 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e8d78d-3bc9-44ac-b061-4f138ded2c78 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:03.379023 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e8d78d-3bc9-44ac-b061-4f138ded2c78 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:03.379295 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e8d78d-3bc9-44ac-b061-4f138ded2c78 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:03.379883 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e773f0-7fd1-44a7-9700-b6ed93021f2e None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:03.386059 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 225/453] 10.222.0.59 () {56 vars in 1374 bytes} [Tue Oct 11 11:47:03 2022] GET /identity/v3/role_assignments?user.id=91a00658b79a42bd91a22a2508b5de34&scope.project.id=26ecf552cc9a4e5ca1d380509683dc4a&role.id=f5a43d8f3fa24a87a7abe7492cf6f87f => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:47:03.386945 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e773f0-7fd1-44a7-9700-b6ed93021f2e None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:03.390412 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e8d78d-3bc9-44ac-b061-4f138ded2c78 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:03.391965 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e8d78d-3bc9-44ac-b061-4f138ded2c78 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:03.413062 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application [None req-19e8d78d-3bc9-44ac-b061-4f138ded2c78 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 11:47:03.413062 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:47:03.413062 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:47:03.413062 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:47:03.413062 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:47:03.413062 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:47:03.413062 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:47:03.413062 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:47:03.413062 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:47:03.413062 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:47:03.413062 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:47:03.413062 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:47:03.413062 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 11:47:03.413062 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 11:47:03.413062 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 11:47:03.413062 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 11:47:03.414015 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:47:03.414015 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:47:03.414015 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:47:03.414015 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:47:03.414015 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:47:03.414015 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:47:03.414015 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:47:03.414015 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application with Lock( Oct 11 11:47:03.414015 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:47:03.414015 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:47:03.414015 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:47:03.414015 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:47:03.414015 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:47:03.414015 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:47:03.414015 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:47:03.414015 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:47:03.414015 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 11:47:03.415309 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 11:47:03.415309 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 11:47:03.415309 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:47:03.415309 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 229/454] 10.222.0.59 () {56 vars in 1108 bytes} [Tue Oct 11 11:47:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:47:03.415309 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e8d78d-3bc9-44ac-b061-4f138ded2c78 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:03.470617 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:03.476624 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:03.524517 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e773f0-7fd1-44a7-9700-b6ed93021f2e None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:03.562870 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e8d78d-3bc9-44ac-b061-4f138ded2c78 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:03.581908 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e946baf-b5b2-4fd3-8367-637a56f5c75b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:03.583011 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e946baf-b5b2-4fd3-8367-637a56f5c75b None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:03.583506 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e946baf-b5b2-4fd3-8367-637a56f5c75b None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:03.583976 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e946baf-b5b2-4fd3-8367-637a56f5c75b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:03.597732 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e946baf-b5b2-4fd3-8367-637a56f5c75b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:03.599656 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e946baf-b5b2-4fd3-8367-637a56f5c75b None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:03.609312 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3f7f93-d723-4ffe-9134-ac47954b4cd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:03.609969 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3f7f93-d723-4ffe-9134-ac47954b4cd0 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:03.610380 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3f7f93-d723-4ffe-9134-ac47954b4cd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:03.610585 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3f7f93-d723-4ffe-9134-ac47954b4cd0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:03.612865 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application [None req-9e946baf-b5b2-4fd3-8367-637a56f5c75b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 11:47:03.612865 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:47:03.612865 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:47:03.612865 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:47:03.612865 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:47:03.612865 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:47:03.612865 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:47:03.612865 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:47:03.612865 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:47:03.612865 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:47:03.612865 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:47:03.612865 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:47:03.612865 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 11 11:47:03.612865 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 11 11:47:03.612865 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 11 11:47:03.613711 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 11 11:47:03.613711 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:47:03.613711 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:47:03.613711 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:47:03.613711 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:47:03.613711 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:47:03.613711 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:47:03.613711 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:47:03.613711 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application with Lock( Oct 11 11:47:03.613711 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:47:03.613711 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:47:03.613711 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:47:03.613711 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:47:03.613711 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:47:03.613711 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:47:03.613711 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:47:03.613711 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:47:03.614686 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 11 11:47:03.614686 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 11 11:47:03.614686 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 11 11:47:03.614686 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 11 11:47:03.614686 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 11 11:47:03.614686 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 11 11:47:03.614686 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 11:47:03.614686 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:47:03.618662 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 226/455] 10.222.0.59 () {56 vars in 1111 bytes} [Tue Oct 11 11:47:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:47:03.620450 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e946baf-b5b2-4fd3-8367-637a56f5c75b None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:03.627871 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3f7f93-d723-4ffe-9134-ac47954b4cd0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:03.629377 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3f7f93-d723-4ffe-9134-ac47954b4cd0 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:03.646922 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application [None req-3f3f7f93-d723-4ffe-9134-ac47954b4cd0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 11:47:03.646922 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:47:03.646922 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:47:03.646922 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:47:03.646922 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:47:03.646922 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:47:03.646922 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:47:03.646922 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:47:03.646922 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:47:03.646922 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:47:03.646922 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:47:03.646922 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:47:03.646922 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 11:47:03.646922 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 11:47:03.646922 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 11:47:03.646922 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 11:47:03.649436 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:47:03.649436 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:47:03.649436 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:47:03.649436 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:47:03.649436 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:47:03.649436 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:47:03.649436 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:47:03.649436 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application with Lock( Oct 11 11:47:03.649436 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:47:03.649436 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:47:03.649436 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:47:03.649436 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:47:03.649436 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:47:03.649436 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:47:03.649436 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:47:03.649436 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:47:03.649436 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 11:47:03.651724 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 11:47:03.651724 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 11:47:03.651724 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:47:03.651724 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 230/456] 10.222.0.59 () {56 vars in 1108 bytes} [Tue Oct 11 11:47:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:47:03.651724 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3f7f93-d723-4ffe-9134-ac47954b4cd0 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:03.770032 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:03.772012 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e946baf-b5b2-4fd3-8367-637a56f5c75b None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:03.777577 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3f7f93-d723-4ffe-9134-ac47954b4cd0 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:03.807682 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:03.825113 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5373dd-c2aa-4415-aebd-24ac30e34475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:03.826164 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5373dd-c2aa-4415-aebd-24ac30e34475 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:03.826552 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5373dd-c2aa-4415-aebd-24ac30e34475 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:03.826916 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5373dd-c2aa-4415-aebd-24ac30e34475 None admin] PATH_INFO: `/v3/roles` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:03.827724 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5373dd-c2aa-4415-aebd-24ac30e34475 None admin] RBAC: Adding query filter params (name=service) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:03.828091 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5373dd-c2aa-4415-aebd-24ac30e34475 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:03.829619 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5373dd-c2aa-4415-aebd-24ac30e34475 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:03.831868 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc02788e-7098-4300-a192-1de0d58fed2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:03.832964 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc02788e-7098-4300-a192-1de0d58fed2f None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:03.833444 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc02788e-7098-4300-a192-1de0d58fed2f None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:03.833916 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc02788e-7098-4300-a192-1de0d58fed2f None admin] PATH_INFO: `/v3/domains` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:03.835160 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc02788e-7098-4300-a192-1de0d58fed2f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:03.835642 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc02788e-7098-4300-a192-1de0d58fed2f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:03.838329 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc02788e-7098-4300-a192-1de0d58fed2f None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:03.842265 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 227/457] 10.222.0.59 () {56 vars in 1103 bytes} [Tue Oct 11 11:47:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:47:03.844314 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5373dd-c2aa-4415-aebd-24ac30e34475 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:03.863358 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 231/458] 10.222.0.59 () {56 vars in 1109 bytes} [Tue Oct 11 11:47:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:47:03.863358 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc02788e-7098-4300-a192-1de0d58fed2f None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:03.996520 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5373dd-c2aa-4415-aebd-24ac30e34475 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:04.024388 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc02788e-7098-4300-a192-1de0d58fed2f None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:04.043853 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474fa19f-aa30-4b8d-a35b-35789d3bafdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:04.044671 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474fa19f-aa30-4b8d-a35b-35789d3bafdc None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:04.045042 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474fa19f-aa30-4b8d-a35b-35789d3bafdc None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:04.045407 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474fa19f-aa30-4b8d-a35b-35789d3bafdc None admin] PATH_INFO: `/v3/projects` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:04.046211 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474fa19f-aa30-4b8d-a35b-35789d3bafdc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:04.046639 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474fa19f-aa30-4b8d-a35b-35789d3bafdc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:04.048324 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474fa19f-aa30-4b8d-a35b-35789d3bafdc None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:04.068186 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 228/459] 10.222.0.59 () {56 vars in 1148 bytes} [Tue Oct 11 11:47:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:47:04.068186 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474fa19f-aa30-4b8d-a35b-35789d3bafdc None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:04.083082 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6299309-0e45-431e-9f2a-fd6063ac331b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:04.083711 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6299309-0e45-431e-9f2a-fd6063ac331b None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:04.083983 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6299309-0e45-431e-9f2a-fd6063ac331b None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:04.084260 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6299309-0e45-431e-9f2a-fd6063ac331b None admin] PATH_INFO: `/v3/domains` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:04.085052 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6299309-0e45-431e-9f2a-fd6063ac331b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:04.085363 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6299309-0e45-431e-9f2a-fd6063ac331b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:04.087086 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6299309-0e45-431e-9f2a-fd6063ac331b None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:04.101950 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 232/460] 10.222.0.59 () {56 vars in 1109 bytes} [Tue Oct 11 11:47:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:47:04.102951 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6299309-0e45-431e-9f2a-fd6063ac331b None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:04.125667 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:04.146246 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:04.223802 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474fa19f-aa30-4b8d-a35b-35789d3bafdc None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:04.248150 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6299309-0e45-431e-9f2a-fd6063ac331b None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:04.275037 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33b9cf5-8657-43a5-9dac-f0199f489978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:04.275967 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33b9cf5-8657-43a5-9dac-f0199f489978 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:04.276769 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33b9cf5-8657-43a5-9dac-f0199f489978 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:04.276924 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33b9cf5-8657-43a5-9dac-f0199f489978 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:04.291287 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350842ea-56ff-40d2-b3f6-f6628a7b7d91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:04.292007 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350842ea-56ff-40d2-b3f6-f6628a7b7d91 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:04.292250 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350842ea-56ff-40d2-b3f6-f6628a7b7d91 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:04.292535 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350842ea-56ff-40d2-b3f6-f6628a7b7d91 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:04.296693 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33b9cf5-8657-43a5-9dac-f0199f489978 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:04.299631 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33b9cf5-8657-43a5-9dac-f0199f489978 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:04.301376 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350842ea-56ff-40d2-b3f6-f6628a7b7d91 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:04.302937 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350842ea-56ff-40d2-b3f6-f6628a7b7d91 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:04.321616 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application [None req-350842ea-56ff-40d2-b3f6-f6628a7b7d91 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 11 11:47:04.321616 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:47:04.321616 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:47:04.321616 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:47:04.321616 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:47:04.321616 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:47:04.321616 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:47:04.321616 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:47:04.321616 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:47:04.321616 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:47:04.321616 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:47:04.321616 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:47:04.321616 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 11 11:47:04.321616 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 11 11:47:04.321616 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 11 11:47:04.321616 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 11 11:47:04.322623 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:47:04.322623 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:47:04.322623 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 11 11:47:04.322623 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 11:47:04.322623 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 11 11:47:04.322623 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 11:47:04.322623 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:47:04.322623 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:47:04.322623 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:47:04.322623 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:47:04.322623 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:47:04.322623 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application with Lock( Oct 11 11:47:04.322623 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:47:04.322623 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:47:04.322623 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:47:04.322623 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:47:04.322623 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:47:04.322623 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:47:04.323534 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:47:04.323534 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:47:04.323534 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 11 11:47:04.323534 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 11 11:47:04.323534 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 11 11:47:04.323534 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 11 11:47:04.323534 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 11 11:47:04.323534 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 11 11:47:04.323534 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 11 11:47:04.323534 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:47:04.324435 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 233/461] 10.222.0.59 () {56 vars in 1093 bytes} [Tue Oct 11 11:47:04 2022] GET /identity/v3/users/nova => generated 82 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:47:04.325389 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350842ea-56ff-40d2-b3f6-f6628a7b7d91 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:04.339901 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application [None req-c33b9cf5-8657-43a5-9dac-f0199f489978 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 11:47:04.339901 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:47:04.339901 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:47:04.339901 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:47:04.339901 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:47:04.339901 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:47:04.339901 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:47:04.339901 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:47:04.339901 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:47:04.339901 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:47:04.339901 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:47:04.339901 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:47:04.339901 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 11:47:04.339901 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 11:47:04.339901 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 11:47:04.339901 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 11:47:04.341673 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:47:04.341673 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:47:04.341673 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:47:04.341673 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:47:04.341673 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:47:04.341673 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:47:04.341673 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:47:04.341673 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application with Lock( Oct 11 11:47:04.341673 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:47:04.341673 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:47:04.341673 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:47:04.341673 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:47:04.341673 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:47:04.341673 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:47:04.341673 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:47:04.341673 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:47:04.341673 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 11:47:04.343197 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 11:47:04.343197 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 11:47:04.343197 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:47:04.343197 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 229/462] 10.222.0.59 () {56 vars in 1108 bytes} [Tue Oct 11 11:47:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:47:04.343197 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33b9cf5-8657-43a5-9dac-f0199f489978 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:04.435719 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:04.470079 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350842ea-56ff-40d2-b3f6-f6628a7b7d91 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:04.502638 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:04.510551 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33b9cf5-8657-43a5-9dac-f0199f489978 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:04.514885 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db81f196-04c4-4a95-b6ee-413389c0602c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:04.516064 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db81f196-04c4-4a95-b6ee-413389c0602c None admin] REQUEST_METHOD: `PUT` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:04.516591 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db81f196-04c4-4a95-b6ee-413389c0602c None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:04.517063 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db81f196-04c4-4a95-b6ee-413389c0602c None admin] PATH_INFO: `/v3/projects/26ecf552cc9a4e5ca1d380509683dc4a/users/76e7fee1993a40d5ba4295e7daeb3e29/roles/f5a43d8f3fa24a87a7abe7492cf6f87f` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:04.548043 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e03a0a-f8f3-46db-b6fb-2ca8e1d92e9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:04.548689 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e03a0a-f8f3-46db-b6fb-2ca8e1d92e9d None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:04.549033 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e03a0a-f8f3-46db-b6fb-2ca8e1d92e9d None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:04.549268 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e03a0a-f8f3-46db-b6fb-2ca8e1d92e9d None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:04.554139 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db81f196-04c4-4a95-b6ee-413389c0602c None admin] RBAC: Authorizing `identity:create_grant(project_id=26ecf552cc9a4e5ca1d380509683dc4a, user_id=76e7fee1993a40d5ba4295e7daeb3e29, role_id=f5a43d8f3fa24a87a7abe7492cf6f87f)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:04.557756 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e03a0a-f8f3-46db-b6fb-2ca8e1d92e9d None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:04.558050 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db81f196-04c4-4a95-b6ee-413389c0602c None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:04.559255 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e03a0a-f8f3-46db-b6fb-2ca8e1d92e9d None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:04.582591 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 234/463] 10.222.0.59 () {58 vars in 1439 bytes} [Tue Oct 11 11:47:04 2022] PUT /identity/v3/projects/26ecf552cc9a4e5ca1d380509683dc4a/users/76e7fee1993a40d5ba4295e7daeb3e29/roles/f5a43d8f3fa24a87a7abe7492cf6f87f => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 11:47:04.582591 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db81f196-04c4-4a95-b6ee-413389c0602c None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:04.582876 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application [None req-15e03a0a-f8f3-46db-b6fb-2ca8e1d92e9d None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 11 11:47:04.582876 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:47:04.582876 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:47:04.582876 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:47:04.582876 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:47:04.582876 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:47:04.582876 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:47:04.582876 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:47:04.582876 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:47:04.582876 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:47:04.582876 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:47:04.582876 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:47:04.582876 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 11 11:47:04.582876 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 11 11:47:04.582876 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 11 11:47:04.582876 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 11 11:47:04.583724 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:47:04.583724 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:47:04.583724 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 11 11:47:04.583724 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 11:47:04.583724 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 11 11:47:04.583724 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 11:47:04.583724 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:47:04.583724 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:47:04.583724 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:47:04.583724 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:47:04.583724 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:47:04.583724 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application with Lock( Oct 11 11:47:04.583724 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:47:04.583724 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:47:04.583724 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:47:04.583724 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:47:04.583724 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:47:04.583724 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:47:04.584621 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:47:04.584621 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:47:04.584621 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 11 11:47:04.584621 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 11 11:47:04.584621 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 11 11:47:04.584621 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 11 11:47:04.584621 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 11 11:47:04.584621 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 11 11:47:04.584621 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 11 11:47:04.584621 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:47:04.588000 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 230/464] 10.222.0.59 () {56 vars in 1102 bytes} [Tue Oct 11 11:47:04 2022] GET /identity/v3/users/neutron => generated 85 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:47:04.589275 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e03a0a-f8f3-46db-b6fb-2ca8e1d92e9d None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:04.809194 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db81f196-04c4-4a95-b6ee-413389c0602c None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:04.809377 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e03a0a-f8f3-46db-b6fb-2ca8e1d92e9d None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:04.846424 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:04.857425 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68b36dc-028c-4093-bb61-7ae174d830a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:04.858326 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68b36dc-028c-4093-bb61-7ae174d830a6 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:04.858691 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68b36dc-028c-4093-bb61-7ae174d830a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:04.858958 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68b36dc-028c-4093-bb61-7ae174d830a6 None admin] PATH_INFO: `/v3/users` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:04.860586 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68b36dc-028c-4093-bb61-7ae174d830a6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:04.860586 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68b36dc-028c-4093-bb61-7ae174d830a6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:04.862311 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68b36dc-028c-4093-bb61-7ae174d830a6 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:04.887842 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ba4748-6800-4b29-bbeb-b05967df8afd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:04.888696 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 235/465] 10.222.0.59 () {56 vars in 1133 bytes} [Tue Oct 11 11:47:04 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:47:04.889463 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ba4748-6800-4b29-bbeb-b05967df8afd None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:04.889748 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68b36dc-028c-4093-bb61-7ae174d830a6 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:04.890396 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ba4748-6800-4b29-bbeb-b05967df8afd None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:04.891076 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ba4748-6800-4b29-bbeb-b05967df8afd None admin] PATH_INFO: `/v3/domains` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:04.892452 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ba4748-6800-4b29-bbeb-b05967df8afd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:04.893395 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ba4748-6800-4b29-bbeb-b05967df8afd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:04.896434 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ba4748-6800-4b29-bbeb-b05967df8afd None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:04.924403 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 231/466] 10.222.0.59 () {56 vars in 1109 bytes} [Tue Oct 11 11:47:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:47:04.931648 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ba4748-6800-4b29-bbeb-b05967df8afd None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:04.973941 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:05.086716 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68b36dc-028c-4093-bb61-7ae174d830a6 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:05.140688 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ba4748-6800-4b29-bbeb-b05967df8afd None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:05.151490 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52e7305-5651-4797-be13-e340a3d53a8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:05.152684 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52e7305-5651-4797-be13-e340a3d53a8b None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:05.153145 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52e7305-5651-4797-be13-e340a3d53a8b None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:05.153623 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52e7305-5651-4797-be13-e340a3d53a8b None admin] PATH_INFO: `/v3/users` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:05.155040 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52e7305-5651-4797-be13-e340a3d53a8b None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:05.155596 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52e7305-5651-4797-be13-e340a3d53a8b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:05.158883 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52e7305-5651-4797-be13-e340a3d53a8b None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:05.192800 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 236/467] 10.222.0.59 () {56 vars in 1139 bytes} [Tue Oct 11 11:47:04 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:47:05.194162 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52e7305-5651-4797-be13-e340a3d53a8b None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:05.211705 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e416f856-cb54-45c9-8f55-1ca267946e57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:05.213127 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e416f856-cb54-45c9-8f55-1ca267946e57 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:05.214052 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e416f856-cb54-45c9-8f55-1ca267946e57 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:05.214869 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e416f856-cb54-45c9-8f55-1ca267946e57 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:05.269015 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:05.357477 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e416f856-cb54-45c9-8f55-1ca267946e57 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:05.361028 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e416f856-cb54-45c9-8f55-1ca267946e57 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:05.376730 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52e7305-5651-4797-be13-e340a3d53a8b None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:05.394526 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:05.444765 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccdc519-200c-4386-aefe-89865999075d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:05.445986 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccdc519-200c-4386-aefe-89865999075d None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:05.446488 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccdc519-200c-4386-aefe-89865999075d None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:05.447078 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccdc519-200c-4386-aefe-89865999075d None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:05.462218 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccdc519-200c-4386-aefe-89865999075d None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:05.464954 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccdc519-200c-4386-aefe-89865999075d None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:05.492631 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application [None req-9ccdc519-200c-4386-aefe-89865999075d None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 11 11:47:05.492631 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:47:05.492631 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:47:05.492631 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:47:05.492631 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:47:05.492631 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:47:05.492631 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:47:05.492631 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:47:05.492631 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:47:05.492631 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:47:05.492631 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:47:05.492631 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:47:05.492631 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 11 11:47:05.492631 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 11 11:47:05.492631 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 11 11:47:05.492631 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 11 11:47:05.494338 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:47:05.494338 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:47:05.494338 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 11 11:47:05.494338 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 11:47:05.494338 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 11 11:47:05.494338 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 11:47:05.494338 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:47:05.494338 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:47:05.494338 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:47:05.494338 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:47:05.494338 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:47:05.494338 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application with Lock( Oct 11 11:47:05.494338 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:47:05.494338 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:47:05.494338 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:47:05.494338 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:47:05.494338 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:47:05.494338 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:47:05.496096 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:47:05.496096 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:47:05.496096 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 11 11:47:05.496096 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 11 11:47:05.496096 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 11 11:47:05.496096 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 11 11:47:05.496096 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 11 11:47:05.496096 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 11 11:47:05.496096 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 11 11:47:05.496096 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:47:05.502419 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 237/468] 10.222.0.59 () {56 vars in 1099 bytes} [Tue Oct 11 11:47:05 2022] GET /identity/v3/users/cinder => generated 84 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:47:05.504138 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccdc519-200c-4386-aefe-89865999075d None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:05.571412 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 232/469] 10.222.0.59 () {58 vars in 1282 bytes} [Tue Oct 11 11:47:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 646 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 11 11:47:05.573649 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e416f856-cb54-45c9-8f55-1ca267946e57 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:05.706517 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccdc519-200c-4386-aefe-89865999075d None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:05.709417 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:05.761536 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e416f856-cb54-45c9-8f55-1ca267946e57 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:05.766898 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f7477c-e2db-41bb-95ad-26bd47a6291c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:05.768000 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f7477c-e2db-41bb-95ad-26bd47a6291c None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:05.768503 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f7477c-e2db-41bb-95ad-26bd47a6291c None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:05.769099 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f7477c-e2db-41bb-95ad-26bd47a6291c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:05.792846 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:05.823063 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68225318-3547-4d6e-9082-f55c73c71343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:05.824111 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68225318-3547-4d6e-9082-f55c73c71343 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:05.824544 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68225318-3547-4d6e-9082-f55c73c71343 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:05.825023 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68225318-3547-4d6e-9082-f55c73c71343 None admin] PATH_INFO: `/v3/users` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:05.826380 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68225318-3547-4d6e-9082-f55c73c71343 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:05.826974 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68225318-3547-4d6e-9082-f55c73c71343 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:05.829723 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68225318-3547-4d6e-9082-f55c73c71343 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:05.872037 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 233/470] 10.222.0.59 () {56 vars in 1137 bytes} [Tue Oct 11 11:47:05 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:47:05.872037 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68225318-3547-4d6e-9082-f55c73c71343 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:05.888608 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f7477c-e2db-41bb-95ad-26bd47a6291c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:05.890226 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f7477c-e2db-41bb-95ad-26bd47a6291c None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:06.041454 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68225318-3547-4d6e-9082-f55c73c71343 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:06.072700 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 238/471] 10.222.0.59 () {58 vars in 1282 bytes} [Tue Oct 11 11:47:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 564 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 11 11:47:06.072700 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25adc603-394b-4eed-a8cd-47fa05b4388c None None] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:06.072700 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25adc603-394b-4eed-a8cd-47fa05b4388c None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:06.072700 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25adc603-394b-4eed-a8cd-47fa05b4388c None None] PATH_INFO: `/` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:06.072700 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 239/472] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:47:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:47:06.072700 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25adc603-394b-4eed-a8cd-47fa05b4388c None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:06.078145 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:06.117616 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf185bba-f238-4695-b85b-adede83b85de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:06.118698 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf185bba-f238-4695-b85b-adede83b85de None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:06.119273 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf185bba-f238-4695-b85b-adede83b85de None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:06.119983 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf185bba-f238-4695-b85b-adede83b85de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:06.139205 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf185bba-f238-4695-b85b-adede83b85de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:06.141977 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf185bba-f238-4695-b85b-adede83b85de None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:06.170928 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application [None req-bf185bba-f238-4695-b85b-adede83b85de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 11:47:06.170928 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:47:06.170928 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:47:06.170928 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:47:06.170928 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:47:06.170928 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:47:06.170928 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:47:06.170928 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:47:06.170928 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:47:06.170928 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:47:06.170928 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:47:06.170928 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:47:06.170928 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 11:47:06.170928 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 11:47:06.170928 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 11:47:06.170928 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 11:47:06.172703 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:47:06.172703 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:47:06.172703 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:47:06.172703 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:47:06.172703 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:47:06.172703 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:47:06.172703 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:47:06.172703 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application with Lock( Oct 11 11:47:06.172703 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:47:06.172703 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:47:06.172703 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:47:06.172703 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:47:06.172703 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:47:06.172703 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:47:06.172703 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:47:06.172703 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:47:06.172703 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 11:47:06.174419 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 11:47:06.174419 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 11:47:06.174419 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:47:06.174419 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 234/473] 10.222.0.59 () {56 vars in 1108 bytes} [Tue Oct 11 11:47:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:47:06.175905 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcfcb13-03ec-404c-a26d-3b68bff45f61 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:06.175905 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcfcb13-03ec-404c-a26d-3b68bff45f61 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:06.176335 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcfcb13-03ec-404c-a26d-3b68bff45f61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:06.206599 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-0fcfcb13-03ec-404c-a26d-3b68bff45f61 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:47:06.209629 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-0fcfcb13-03ec-404c-a26d-3b68bff45f61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:47:06.219804 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25adc603-394b-4eed-a8cd-47fa05b4388c None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:06.249341 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:06.270507 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c702144-cbb2-46ba-98a5-baf6266bcc82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:06.271618 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c702144-cbb2-46ba-98a5-baf6266bcc82 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:06.272094 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c702144-cbb2-46ba-98a5-baf6266bcc82 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:06.272569 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c702144-cbb2-46ba-98a5-baf6266bcc82 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:06.331411 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 235/474] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:47:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 11:47:06.331411 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcfcb13-03ec-404c-a26d-3b68bff45f61 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:06.365063 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c702144-cbb2-46ba-98a5-baf6266bcc82 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:06.367523 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c702144-cbb2-46ba-98a5-baf6266bcc82 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:06.370164 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:06.446158 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcfcb13-03ec-404c-a26d-3b68bff45f61 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:06.485198 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00785308-5207-42ba-a3f5-4b46824ec100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:06.486175 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00785308-5207-42ba-a3f5-4b46824ec100 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:06.486513 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00785308-5207-42ba-a3f5-4b46824ec100 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:06.486824 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00785308-5207-42ba-a3f5-4b46824ec100 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:06.498650 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00785308-5207-42ba-a3f5-4b46824ec100 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:06.500154 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00785308-5207-42ba-a3f5-4b46824ec100 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:06.501307 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 240/475] 10.222.0.59 () {58 vars in 1282 bytes} [Tue Oct 11 11:47:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 11 11:47:06.502529 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c702144-cbb2-46ba-98a5-baf6266bcc82 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:06.517342 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application [None req-00785308-5207-42ba-a3f5-4b46824ec100 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 11:47:06.517342 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:47:06.517342 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:47:06.517342 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:47:06.517342 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:47:06.517342 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:47:06.517342 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:47:06.517342 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:47:06.517342 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:47:06.517342 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:47:06.517342 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:47:06.517342 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:47:06.517342 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 11:47:06.517342 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 11:47:06.517342 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 11:47:06.517342 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 11:47:06.519133 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:47:06.519133 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:47:06.519133 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:47:06.519133 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:47:06.519133 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:47:06.519133 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:47:06.519133 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:47:06.519133 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application with Lock( Oct 11 11:47:06.519133 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:47:06.519133 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:47:06.519133 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:47:06.519133 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:47:06.519133 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:47:06.519133 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:47:06.519133 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:47:06.519133 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:47:06.519133 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 11:47:06.520777 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 11:47:06.520777 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 11:47:06.520777 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:47:06.525271 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 236/476] 10.222.0.59 () {56 vars in 1108 bytes} [Tue Oct 11 11:47:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:47:06.525271 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00785308-5207-42ba-a3f5-4b46824ec100 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:06.537664 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:06.618617 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c702144-cbb2-46ba-98a5-baf6266bcc82 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:06.620844 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:06.643334 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00785308-5207-42ba-a3f5-4b46824ec100 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:06.658979 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8efb1bc-b4a9-42c6-91f8-3480dd1dfb84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:06.660234 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8efb1bc-b4a9-42c6-91f8-3480dd1dfb84 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:06.660664 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8efb1bc-b4a9-42c6-91f8-3480dd1dfb84 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:06.661071 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8efb1bc-b4a9-42c6-91f8-3480dd1dfb84 None admin] PATH_INFO: `/v3/domains` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:06.661955 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8efb1bc-b4a9-42c6-91f8-3480dd1dfb84 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:06.662390 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8efb1bc-b4a9-42c6-91f8-3480dd1dfb84 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:06.664130 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8efb1bc-b4a9-42c6-91f8-3480dd1dfb84 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:06.681997 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 241/477] 10.222.0.59 () {56 vars in 1109 bytes} [Tue Oct 11 11:47:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:47:06.681997 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8efb1bc-b4a9-42c6-91f8-3480dd1dfb84 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:06.696402 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19698cc8-bc2b-4f8d-9722-492b212c0637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:06.698619 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19698cc8-bc2b-4f8d-9722-492b212c0637 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:06.699394 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19698cc8-bc2b-4f8d-9722-492b212c0637 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:06.700428 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19698cc8-bc2b-4f8d-9722-492b212c0637 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:06.723082 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19698cc8-bc2b-4f8d-9722-492b212c0637 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:06.725533 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19698cc8-bc2b-4f8d-9722-492b212c0637 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:06.748281 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application [None req-19698cc8-bc2b-4f8d-9722-492b212c0637 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 11:47:06.748281 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:47:06.748281 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:47:06.748281 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:47:06.748281 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:47:06.748281 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:47:06.748281 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:47:06.748281 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:47:06.748281 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:47:06.748281 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:47:06.748281 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:47:06.748281 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:47:06.748281 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 11:47:06.748281 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 11:47:06.748281 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 11:47:06.748281 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 11:47:06.750185 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:47:06.750185 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:47:06.750185 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:47:06.750185 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:47:06.750185 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:47:06.750185 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:47:06.750185 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:47:06.750185 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application with Lock( Oct 11 11:47:06.750185 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:47:06.750185 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:47:06.750185 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:47:06.750185 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:47:06.750185 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:47:06.750185 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:47:06.750185 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:47:06.750185 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:47:06.750185 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 11:47:06.751796 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 11:47:06.751796 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 11:47:06.751796 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:47:06.751796 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 237/478] 10.222.0.59 () {56 vars in 1108 bytes} [Tue Oct 11 11:47:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:47:06.751796 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19698cc8-bc2b-4f8d-9722-492b212c0637 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:06.818262 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8efb1bc-b4a9-42c6-91f8-3480dd1dfb84 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:06.835274 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:06.867731 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92375f58-dcf6-41b7-ad1c-c726f1e09c5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:06.868353 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92375f58-dcf6-41b7-ad1c-c726f1e09c5b None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:06.868628 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92375f58-dcf6-41b7-ad1c-c726f1e09c5b None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:06.868904 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92375f58-dcf6-41b7-ad1c-c726f1e09c5b None admin] PATH_INFO: `/v3/domains` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:06.869565 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92375f58-dcf6-41b7-ad1c-c726f1e09c5b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:06.869855 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92375f58-dcf6-41b7-ad1c-c726f1e09c5b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:06.871327 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92375f58-dcf6-41b7-ad1c-c726f1e09c5b None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:06.894827 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 242/479] 10.222.0.59 () {56 vars in 1109 bytes} [Tue Oct 11 11:47:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:47:06.894827 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92375f58-dcf6-41b7-ad1c-c726f1e09c5b None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:06.911684 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19698cc8-bc2b-4f8d-9722-492b212c0637 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:06.929092 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:06.983080 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2609db36-6f74-4604-99be-2b42f734de18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:06.984176 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2609db36-6f74-4604-99be-2b42f734de18 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:06.984611 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2609db36-6f74-4604-99be-2b42f734de18 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:06.985058 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2609db36-6f74-4604-99be-2b42f734de18 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:06.997697 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2609db36-6f74-4604-99be-2b42f734de18 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:07.000722 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2609db36-6f74-4604-99be-2b42f734de18 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:07.020772 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application [None req-2609db36-6f74-4604-99be-2b42f734de18 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 11:47:07.020772 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:47:07.020772 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:47:07.020772 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:47:07.020772 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:47:07.020772 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:47:07.020772 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:47:07.020772 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:47:07.020772 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:47:07.020772 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:47:07.020772 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:47:07.020772 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:47:07.020772 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 11 11:47:07.020772 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 11 11:47:07.020772 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 11 11:47:07.022451 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 11 11:47:07.022451 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:47:07.022451 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:47:07.022451 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:47:07.022451 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:47:07.022451 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:47:07.022451 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:47:07.022451 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:47:07.022451 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application with Lock( Oct 11 11:47:07.022451 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:47:07.022451 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:47:07.022451 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:47:07.022451 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:47:07.022451 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:47:07.022451 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:47:07.022451 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:47:07.022451 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:47:07.024485 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 11 11:47:07.024485 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 11 11:47:07.024485 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 11 11:47:07.024485 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 11 11:47:07.024485 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 11 11:47:07.024485 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 11 11:47:07.024485 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 11:47:07.024485 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:47:07.024485 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 238/480] 10.222.0.59 () {56 vars in 1111 bytes} [Tue Oct 11 11:47:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:47:07.024485 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2609db36-6f74-4604-99be-2b42f734de18 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:07.035311 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92375f58-dcf6-41b7-ad1c-c726f1e09c5b None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:07.083927 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc68a13-4a98-4fb1-b62c-faf5f40d643a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:07.085013 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc68a13-4a98-4fb1-b62c-faf5f40d643a None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:07.085503 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc68a13-4a98-4fb1-b62c-faf5f40d643a None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:07.085996 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc68a13-4a98-4fb1-b62c-faf5f40d643a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:07.103960 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc68a13-4a98-4fb1-b62c-faf5f40d643a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:07.106651 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc68a13-4a98-4fb1-b62c-faf5f40d643a None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:07.134348 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application [None req-7fc68a13-4a98-4fb1-b62c-faf5f40d643a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 11:47:07.134348 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:47:07.134348 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:47:07.134348 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:47:07.134348 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:47:07.134348 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:47:07.134348 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:47:07.134348 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:47:07.134348 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:47:07.134348 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:47:07.134348 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:47:07.134348 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:47:07.134348 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 11:47:07.134348 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 11:47:07.134348 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 11:47:07.134348 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 11:47:07.135830 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:47:07.135830 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:47:07.135830 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:47:07.135830 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:47:07.135830 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:47:07.135830 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:47:07.135830 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:47:07.135830 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application with Lock( Oct 11 11:47:07.135830 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:47:07.135830 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:47:07.135830 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:47:07.135830 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:47:07.135830 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:47:07.135830 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:47:07.135830 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:47:07.135830 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:47:07.135830 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 11:47:07.137302 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 11:47:07.137302 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 11:47:07.137302 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:47:07.137302 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 243/481] 10.222.0.59 () {56 vars in 1108 bytes} [Tue Oct 11 11:47:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:47:07.138618 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc68a13-4a98-4fb1-b62c-faf5f40d643a None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:07.141156 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2609db36-6f74-4604-99be-2b42f734de18 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:07.164084 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:07.181597 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87840bd4-f37f-4d04-b71e-60d9450d9446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:07.182221 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87840bd4-f37f-4d04-b71e-60d9450d9446 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:07.182489 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87840bd4-f37f-4d04-b71e-60d9450d9446 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:07.182817 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87840bd4-f37f-4d04-b71e-60d9450d9446 None admin] PATH_INFO: `/v3/domains` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:07.183461 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87840bd4-f37f-4d04-b71e-60d9450d9446 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:07.183888 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87840bd4-f37f-4d04-b71e-60d9450d9446 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:07.185270 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87840bd4-f37f-4d04-b71e-60d9450d9446 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:07.197989 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 239/482] 10.222.0.59 () {56 vars in 1109 bytes} [Tue Oct 11 11:47:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:47:07.199323 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc20734-56b8-41c2-ab3a-0fecc1cefdc8 None None] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:07.199589 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc20734-56b8-41c2-ab3a-0fecc1cefdc8 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:07.199937 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc20734-56b8-41c2-ab3a-0fecc1cefdc8 None None] PATH_INFO: `/` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:07.201167 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 240/483] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:47:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:47:07.201872 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc20734-56b8-41c2-ab3a-0fecc1cefdc8 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:07.239878 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:07.266098 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc68a13-4a98-4fb1-b62c-faf5f40d643a None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:07.304068 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c11e86b-10bc-41ce-9b77-599ddce139f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:07.304705 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c11e86b-10bc-41ce-9b77-599ddce139f6 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:07.305014 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c11e86b-10bc-41ce-9b77-599ddce139f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:07.305285 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c11e86b-10bc-41ce-9b77-599ddce139f6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:07.312808 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c11e86b-10bc-41ce-9b77-599ddce139f6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:07.314008 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c11e86b-10bc-41ce-9b77-599ddce139f6 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:07.326433 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application [None req-4c11e86b-10bc-41ce-9b77-599ddce139f6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 11:47:07.326433 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:47:07.326433 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:47:07.326433 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:47:07.326433 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:47:07.326433 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:47:07.326433 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:47:07.326433 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:47:07.326433 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:47:07.326433 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:47:07.326433 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:47:07.326433 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:47:07.326433 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 11 11:47:07.326433 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 11 11:47:07.326433 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 11 11:47:07.327683 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 11 11:47:07.327683 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:47:07.327683 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:47:07.327683 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:47:07.327683 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:47:07.327683 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:47:07.327683 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:47:07.327683 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:47:07.327683 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application with Lock( Oct 11 11:47:07.327683 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:47:07.327683 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:47:07.327683 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:47:07.327683 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:47:07.327683 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:47:07.327683 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:47:07.327683 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:47:07.327683 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:47:07.328578 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 11 11:47:07.328578 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 11 11:47:07.328578 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 11 11:47:07.328578 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 11 11:47:07.328578 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 11 11:47:07.328578 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 11 11:47:07.328578 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 11:47:07.328578 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:47:07.328578 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 244/484] 10.222.0.59 () {56 vars in 1111 bytes} [Tue Oct 11 11:47:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:47:07.328578 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c11e86b-10bc-41ce-9b77-599ddce139f6 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:07.332639 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc20734-56b8-41c2-ab3a-0fecc1cefdc8 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:07.373694 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1ef62d-15af-4aa1-9970-d6267c2469b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:07.374607 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1ef62d-15af-4aa1-9970-d6267c2469b9 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:07.374883 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1ef62d-15af-4aa1-9970-d6267c2469b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:07.375335 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1ef62d-15af-4aa1-9970-d6267c2469b9 None admin] PATH_INFO: `/v3/projects` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:07.376143 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1ef62d-15af-4aa1-9970-d6267c2469b9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:07.376511 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1ef62d-15af-4aa1-9970-d6267c2469b9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:07.378061 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1ef62d-15af-4aa1-9970-d6267c2469b9 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:07.391179 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 241/485] 10.222.0.59 () {56 vars in 1148 bytes} [Tue Oct 11 11:47:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:47:07.392038 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1ef62d-15af-4aa1-9970-d6267c2469b9 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:07.437676 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c11e86b-10bc-41ce-9b77-599ddce139f6 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:07.446407 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:07.478872 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95c521e-765d-4b3b-9fc5-3c9c335427f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:07.479494 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95c521e-765d-4b3b-9fc5-3c9c335427f4 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:07.479776 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95c521e-765d-4b3b-9fc5-3c9c335427f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:07.480053 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95c521e-765d-4b3b-9fc5-3c9c335427f4 None admin] PATH_INFO: `/v3/domains` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:07.480706 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95c521e-765d-4b3b-9fc5-3c9c335427f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:07.481010 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95c521e-765d-4b3b-9fc5-3c9c335427f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:07.482431 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95c521e-765d-4b3b-9fc5-3c9c335427f4 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:07.484239 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:07.496474 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 245/486] 10.222.0.59 () {56 vars in 1109 bytes} [Tue Oct 11 11:47:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:47:07.498234 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599bac49-e055-4db9-9323-c34b708d12f8 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:07.498626 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599bac49-e055-4db9-9323-c34b708d12f8 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:07.498926 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599bac49-e055-4db9-9323-c34b708d12f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:07.518215 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-599bac49-e055-4db9-9323-c34b708d12f8 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:47:07.519391 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-599bac49-e055-4db9-9323-c34b708d12f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:47:07.522713 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1ef62d-15af-4aa1-9970-d6267c2469b9 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:07.558250 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fb51bf-a8c9-43d4-8293-fa1fa1ed8a63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:07.558918 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fb51bf-a8c9-43d4-8293-fa1fa1ed8a63 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:07.559198 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fb51bf-a8c9-43d4-8293-fa1fa1ed8a63 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:07.559466 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fb51bf-a8c9-43d4-8293-fa1fa1ed8a63 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:07.566484 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fb51bf-a8c9-43d4-8293-fa1fa1ed8a63 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:07.567965 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fb51bf-a8c9-43d4-8293-fa1fa1ed8a63 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:07.579529 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application [None req-77fb51bf-a8c9-43d4-8293-fa1fa1ed8a63 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 11:47:07.579529 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:47:07.579529 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:47:07.579529 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:47:07.579529 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:47:07.579529 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:47:07.579529 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:47:07.579529 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:47:07.579529 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:47:07.579529 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:47:07.579529 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:47:07.579529 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:47:07.579529 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 11 11:47:07.579529 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 11 11:47:07.579529 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 11 11:47:07.581586 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 11 11:47:07.581586 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:47:07.581586 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:47:07.581586 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:47:07.581586 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:47:07.581586 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:47:07.581586 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:47:07.581586 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:47:07.581586 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application with Lock( Oct 11 11:47:07.581586 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:47:07.581586 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:47:07.581586 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:47:07.581586 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:47:07.581586 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:47:07.581586 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:47:07.581586 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:47:07.581586 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:47:07.584195 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 11 11:47:07.584195 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 11 11:47:07.584195 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 11 11:47:07.584195 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 11 11:47:07.584195 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 11 11:47:07.584195 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 11 11:47:07.584195 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 11:47:07.584195 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:47:07.584195 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 242/487] 10.222.0.59 () {56 vars in 1111 bytes} [Tue Oct 11 11:47:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:47:07.584195 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fb51bf-a8c9-43d4-8293-fa1fa1ed8a63 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:07.616544 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 246/488] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:47:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 11:47:07.617437 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599bac49-e055-4db9-9323-c34b708d12f8 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:07.725983 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:07.729666 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fb51bf-a8c9-43d4-8293-fa1fa1ed8a63 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:07.734723 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599bac49-e055-4db9-9323-c34b708d12f8 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:07.762651 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:07.772422 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6149fa1b-fbb8-4d14-b0e8-03625128e5be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:07.773037 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6149fa1b-fbb8-4d14-b0e8-03625128e5be None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:07.773315 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6149fa1b-fbb8-4d14-b0e8-03625128e5be None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:07.773585 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6149fa1b-fbb8-4d14-b0e8-03625128e5be None admin] PATH_INFO: `/v3/projects` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:07.774284 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6149fa1b-fbb8-4d14-b0e8-03625128e5be None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:07.774615 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6149fa1b-fbb8-4d14-b0e8-03625128e5be None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:07.776054 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6149fa1b-fbb8-4d14-b0e8-03625128e5be None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:07.777761 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53fa16f-c8a7-43e7-9c44-a7ca62dd48de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:07.778499 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53fa16f-c8a7-43e7-9c44-a7ca62dd48de None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:07.778837 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53fa16f-c8a7-43e7-9c44-a7ca62dd48de None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:07.779450 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53fa16f-c8a7-43e7-9c44-a7ca62dd48de None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:07.780064 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53fa16f-c8a7-43e7-9c44-a7ca62dd48de None admin] RBAC: Adding query filter params (role.id=f5a43d8f3fa24a87a7abe7492cf6f87f, scope.project.id=26ecf552cc9a4e5ca1d380509683dc4a, user.id=bad5e1cb306b46818096940acc3810b2) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:07.780404 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53fa16f-c8a7-43e7-9c44-a7ca62dd48de None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:07.782285 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53fa16f-c8a7-43e7-9c44-a7ca62dd48de None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:07.798924 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 243/489] 10.222.0.59 () {56 vars in 1148 bytes} [Tue Oct 11 11:47:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:47:07.798924 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6149fa1b-fbb8-4d14-b0e8-03625128e5be None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:07.799241 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 247/490] 10.222.0.59 () {56 vars in 1374 bytes} [Tue Oct 11 11:47:07 2022] GET /identity/v3/role_assignments?user.id=bad5e1cb306b46818096940acc3810b2&scope.project.id=26ecf552cc9a4e5ca1d380509683dc4a&role.id=f5a43d8f3fa24a87a7abe7492cf6f87f => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:47:07.799241 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53fa16f-c8a7-43e7-9c44-a7ca62dd48de None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:07.948834 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6149fa1b-fbb8-4d14-b0e8-03625128e5be None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:07.951211 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53fa16f-c8a7-43e7-9c44-a7ca62dd48de None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:07.998804 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba4a21c-436e-45fd-89e9-3cc767c42db1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:07.999294 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba4a21c-436e-45fd-89e9-3cc767c42db1 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:07.999574 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba4a21c-436e-45fd-89e9-3cc767c42db1 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:07.999845 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba4a21c-436e-45fd-89e9-3cc767c42db1 None admin] PATH_INFO: `/v3/projects` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:08.000549 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba4a21c-436e-45fd-89e9-3cc767c42db1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:08.000926 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba4a21c-436e-45fd-89e9-3cc767c42db1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:08.002445 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba4a21c-436e-45fd-89e9-3cc767c42db1 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:08.016223 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 248/491] 10.222.0.59 () {56 vars in 1148 bytes} [Tue Oct 11 11:47:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:47:08.016882 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba4a21c-436e-45fd-89e9-3cc767c42db1 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:08.022241 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9473cdba-ac4a-415b-b6f4-82ad6bbf6e46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:08.022981 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9473cdba-ac4a-415b-b6f4-82ad6bbf6e46 None admin] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:08.023230 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9473cdba-ac4a-415b-b6f4-82ad6bbf6e46 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:08.023464 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9473cdba-ac4a-415b-b6f4-82ad6bbf6e46 None admin] PATH_INFO: `/v3/users` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:08.024148 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9473cdba-ac4a-415b-b6f4-82ad6bbf6e46 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:08.025581 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9473cdba-ac4a-415b-b6f4-82ad6bbf6e46 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:08.054660 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 244/492] 10.222.0.59 () {60 vars in 1132 bytes} [Tue Oct 11 11:47:07 2022] POST /identity/v3/users => generated 254 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 11:47:08.059944 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9473cdba-ac4a-415b-b6f4-82ad6bbf6e46 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:08.076092 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:08.085960 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:08.196891 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba4a21c-436e-45fd-89e9-3cc767c42db1 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:08.235361 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9473cdba-ac4a-415b-b6f4-82ad6bbf6e46 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:08.271170 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6aab7d-26f6-45da-967f-5963cd68f3e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:08.272202 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6aab7d-26f6-45da-967f-5963cd68f3e8 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:08.272691 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6aab7d-26f6-45da-967f-5963cd68f3e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:08.273270 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6aab7d-26f6-45da-967f-5963cd68f3e8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:08.274648 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6aab7d-26f6-45da-967f-5963cd68f3e8 None admin] RBAC: Adding query filter params (role.id=f5a43d8f3fa24a87a7abe7492cf6f87f, scope.project.id=26ecf552cc9a4e5ca1d380509683dc4a, user.id=2e563fe174db4cc19ae6bf8162cc3d23) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:08.275181 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6aab7d-26f6-45da-967f-5963cd68f3e8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:08.278026 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6aab7d-26f6-45da-967f-5963cd68f3e8 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:08.292935 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 249/493] 10.222.0.59 () {56 vars in 1374 bytes} [Tue Oct 11 11:47:08 2022] GET /identity/v3/role_assignments?user.id=2e563fe174db4cc19ae6bf8162cc3d23&scope.project.id=26ecf552cc9a4e5ca1d380509683dc4a&role.id=f5a43d8f3fa24a87a7abe7492cf6f87f => generated 608 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:47:08.294501 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6aab7d-26f6-45da-967f-5963cd68f3e8 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:08.323306 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd6d67b-fc61-445d-b30a-9c9acc590419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:08.324397 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd6d67b-fc61-445d-b30a-9c9acc590419 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:08.324841 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd6d67b-fc61-445d-b30a-9c9acc590419 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:08.325300 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd6d67b-fc61-445d-b30a-9c9acc590419 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:08.352232 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd6d67b-fc61-445d-b30a-9c9acc590419 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 11:47:08.361357 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application [None req-4cd6d67b-fc61-445d-b30a-9c9acc590419 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 11 11:47:08.361357 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:47:08.361357 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:47:08.361357 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:47:08.361357 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:47:08.361357 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:47:08.361357 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:47:08.361357 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:47:08.361357 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:47:08.361357 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:47:08.361357 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:47:08.361357 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:47:08.361357 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 11:47:08.361357 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 11:47:08.361357 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 11:47:08.361357 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 11:47:08.362980 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 11:47:08.362980 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 11:47:08.362980 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 11:47:08.362980 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 11:47:08.362980 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:47:08.362980 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:47:08.362980 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:47:08.362980 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:47:08.362980 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:47:08.362980 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:47:08.362980 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:47:08.362980 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application with Lock( Oct 11 11:47:08.362980 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:47:08.362980 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:47:08.362980 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:47:08.362980 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:47:08.363968 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:47:08.363968 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:47:08.363968 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:47:08.363968 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:47:08.363968 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 11:47:08.363968 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 11:47:08.363968 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 11:47:08.363968 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 11:47:08.363968 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 11:47:08.363968 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 11:47:08.363968 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 11 11:47:08.363968 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:47:08.365067 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 245/494] 10.222.0.59 () {56 vars in 1102 bytes} [Tue Oct 11 11:47:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:47:08.374046 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd6d67b-fc61-445d-b30a-9c9acc590419 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:08.521398 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6aab7d-26f6-45da-967f-5963cd68f3e8 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:08.552078 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:08.573204 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:08.596260 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f79a52d-faa6-4df7-8dde-0f67e2bfa104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:08.597953 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f79a52d-faa6-4df7-8dde-0f67e2bfa104 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:08.598597 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f79a52d-faa6-4df7-8dde-0f67e2bfa104 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:08.599328 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f79a52d-faa6-4df7-8dde-0f67e2bfa104 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:08.602986 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f79a52d-faa6-4df7-8dde-0f67e2bfa104 None admin] RBAC: Adding query filter params (role.id=f5a43d8f3fa24a87a7abe7492cf6f87f, scope.project.id=26ecf552cc9a4e5ca1d380509683dc4a, user.id=2b1b651c73664d08accc6edf2f79bace) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:08.603567 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f79a52d-faa6-4df7-8dde-0f67e2bfa104 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:08.606456 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd6d67b-fc61-445d-b30a-9c9acc590419 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:08.606800 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f79a52d-faa6-4df7-8dde-0f67e2bfa104 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:08.621331 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 250/495] 10.222.0.59 () {56 vars in 1374 bytes} [Tue Oct 11 11:47:08 2022] GET /identity/v3/role_assignments?user.id=2b1b651c73664d08accc6edf2f79bace&scope.project.id=26ecf552cc9a4e5ca1d380509683dc4a&role.id=f5a43d8f3fa24a87a7abe7492cf6f87f => generated 608 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:47:08.662722 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804b4f40-9b0a-442c-9e60-7cc5e9a1b858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:08.663836 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804b4f40-9b0a-442c-9e60-7cc5e9a1b858 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:08.664381 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804b4f40-9b0a-442c-9e60-7cc5e9a1b858 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:08.664680 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804b4f40-9b0a-442c-9e60-7cc5e9a1b858 None admin] PATH_INFO: `/v3/roles` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:08.665387 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804b4f40-9b0a-442c-9e60-7cc5e9a1b858 None admin] RBAC: Adding query filter params (name=service) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:08.665706 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804b4f40-9b0a-442c-9e60-7cc5e9a1b858 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:08.667418 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804b4f40-9b0a-442c-9e60-7cc5e9a1b858 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:08.679800 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 246/496] 10.222.0.59 () {56 vars in 1103 bytes} [Tue Oct 11 11:47:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 11:47:08.686345 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f79a52d-faa6-4df7-8dde-0f67e2bfa104 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:08.878716 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f79a52d-faa6-4df7-8dde-0f67e2bfa104 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:08.944377 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca92be11-fa30-4237-9706-24a7846f473d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:08.945445 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca92be11-fa30-4237-9706-24a7846f473d None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:08.945919 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca92be11-fa30-4237-9706-24a7846f473d None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:08.946453 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca92be11-fa30-4237-9706-24a7846f473d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:08.967201 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca92be11-fa30-4237-9706-24a7846f473d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:08.969954 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca92be11-fa30-4237-9706-24a7846f473d None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:09.003919 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application [None req-ca92be11-fa30-4237-9706-24a7846f473d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 11:47:09.003919 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:47:09.003919 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:47:09.003919 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:47:09.003919 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:47:09.003919 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:47:09.003919 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:47:09.003919 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:47:09.003919 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:47:09.003919 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:47:09.003919 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:47:09.003919 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:47:09.003919 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 11:47:09.003919 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 11:47:09.003919 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 11:47:09.003919 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 11:47:09.005037 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:47:09.005037 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:47:09.005037 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:47:09.005037 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:47:09.005037 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:47:09.005037 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:47:09.005037 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:47:09.005037 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application with Lock( Oct 11 11:47:09.005037 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:47:09.005037 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:47:09.005037 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:47:09.005037 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:47:09.005037 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:47:09.005037 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:47:09.005037 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:47:09.005037 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:47:09.005037 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 11:47:09.005896 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 11:47:09.005896 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 11:47:09.005896 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:47:09.006632 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 251/497] 10.222.0.59 () {56 vars in 1108 bytes} [Tue Oct 11 11:47:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 11 11:47:09.016615 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804b4f40-9b0a-442c-9e60-7cc5e9a1b858 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:09.222484 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804b4f40-9b0a-442c-9e60-7cc5e9a1b858 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:09.292699 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dbc5dd-36ea-4010-a6b4-e65c296b52bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:09.293730 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dbc5dd-36ea-4010-a6b4-e65c296b52bf None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:09.294224 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dbc5dd-36ea-4010-a6b4-e65c296b52bf None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:09.294723 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dbc5dd-36ea-4010-a6b4-e65c296b52bf None admin] PATH_INFO: `/v3/domains` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:09.295907 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dbc5dd-36ea-4010-a6b4-e65c296b52bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:09.296444 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dbc5dd-36ea-4010-a6b4-e65c296b52bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:09.301725 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dbc5dd-36ea-4010-a6b4-e65c296b52bf None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:09.320291 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:09.337710 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 247/498] 10.222.0.59 () {56 vars in 1109 bytes} [Tue Oct 11 11:47:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:47:09.346331 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca92be11-fa30-4237-9706-24a7846f473d None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:09.439009 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:09.624817 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca92be11-fa30-4237-9706-24a7846f473d None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:09.707544 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce55b680-a9aa-4c01-90f4-f1227b3628be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:09.708631 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce55b680-a9aa-4c01-90f4-f1227b3628be None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:09.709096 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce55b680-a9aa-4c01-90f4-f1227b3628be None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:09.709544 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce55b680-a9aa-4c01-90f4-f1227b3628be None admin] PATH_INFO: `/v3/users/swift` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:09.732503 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce55b680-a9aa-4c01-90f4-f1227b3628be None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:09.735291 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce55b680-a9aa-4c01-90f4-f1227b3628be None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:09.764732 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application [None req-ce55b680-a9aa-4c01-90f4-f1227b3628be None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 11 11:47:09.764732 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:47:09.764732 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:47:09.764732 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:47:09.764732 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:47:09.764732 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:47:09.764732 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:47:09.764732 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:47:09.764732 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:47:09.764732 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:47:09.764732 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:47:09.764732 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:47:09.764732 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 11 11:47:09.764732 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 11 11:47:09.764732 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 11 11:47:09.764732 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 11 11:47:09.765806 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:47:09.765806 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:47:09.765806 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 11 11:47:09.765806 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 11:47:09.765806 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 11 11:47:09.765806 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 11:47:09.765806 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:47:09.765806 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:47:09.765806 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:47:09.765806 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:47:09.765806 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:47:09.765806 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application with Lock( Oct 11 11:47:09.765806 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:47:09.765806 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:47:09.765806 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:47:09.765806 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:47:09.765806 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:47:09.765806 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:47:09.766746 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:47:09.766746 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:47:09.766746 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 11 11:47:09.766746 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 11 11:47:09.766746 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 11 11:47:09.766746 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 11 11:47:09.766746 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 11 11:47:09.766746 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 11 11:47:09.766746 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 11 11:47:09.766746 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:47:09.768526 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 252/499] 10.222.0.59 () {56 vars in 1096 bytes} [Tue Oct 11 11:47:09 2022] GET /identity/v3/users/swift => generated 83 bytes in 422 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:47:09.776118 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dbc5dd-36ea-4010-a6b4-e65c296b52bf None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:09.981018 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dbc5dd-36ea-4010-a6b4-e65c296b52bf None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:10.041891 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb8b71f-2152-4356-8623-6911181201cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:10.042939 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb8b71f-2152-4356-8623-6911181201cd None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:10.043390 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb8b71f-2152-4356-8623-6911181201cd None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:10.043809 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb8b71f-2152-4356-8623-6911181201cd None admin] PATH_INFO: `/v3/users` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:10.045009 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb8b71f-2152-4356-8623-6911181201cd None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:10.045486 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb8b71f-2152-4356-8623-6911181201cd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:10.048045 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb8b71f-2152-4356-8623-6911181201cd None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:10.082413 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 248/500] 10.222.0.59 () {56 vars in 1135 bytes} [Tue Oct 11 11:47:09 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 11:47:10.091277 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce55b680-a9aa-4c01-90f4-f1227b3628be None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:10.250829 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:10.271553 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce55b680-a9aa-4c01-90f4-f1227b3628be None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:10.326945 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9faf4fa-e676-409d-9fcc-7d3d71cb0431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:10.327774 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9faf4fa-e676-409d-9fcc-7d3d71cb0431 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:10.328178 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9faf4fa-e676-409d-9fcc-7d3d71cb0431 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:10.328574 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9faf4fa-e676-409d-9fcc-7d3d71cb0431 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:10.428924 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9faf4fa-e676-409d-9fcc-7d3d71cb0431 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:10.431342 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9faf4fa-e676-409d-9fcc-7d3d71cb0431 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:10.534798 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:10.551241 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 253/501] 10.222.0.59 () {58 vars in 1282 bytes} [Tue Oct 11 11:47:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 11 11:47:10.557982 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb8b71f-2152-4356-8623-6911181201cd None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:10.702672 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:10.766248 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb8b71f-2152-4356-8623-6911181201cd None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:10.820846 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f4edd5-152b-47e5-a434-dec128ab9a4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:10.821510 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f4edd5-152b-47e5-a434-dec128ab9a4f None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:10.821784 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f4edd5-152b-47e5-a434-dec128ab9a4f None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:10.822051 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f4edd5-152b-47e5-a434-dec128ab9a4f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:10.838747 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f4edd5-152b-47e5-a434-dec128ab9a4f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:10.840221 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f4edd5-152b-47e5-a434-dec128ab9a4f None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:10.858900 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application [None req-b0f4edd5-152b-47e5-a434-dec128ab9a4f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 11:47:10.858900 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:47:10.858900 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:47:10.858900 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:47:10.858900 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:47:10.858900 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:47:10.858900 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:47:10.858900 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:47:10.858900 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:47:10.858900 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:47:10.858900 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:47:10.858900 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:47:10.858900 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 11:47:10.858900 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 11:47:10.858900 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 11:47:10.858900 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 11:47:10.860362 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:47:10.860362 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:47:10.860362 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:47:10.860362 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:47:10.860362 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:47:10.860362 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:47:10.860362 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:47:10.860362 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application with Lock( Oct 11 11:47:10.860362 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:47:10.860362 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:47:10.860362 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:47:10.860362 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:47:10.860362 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:47:10.860362 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:47:10.860362 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:47:10.860362 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:47:10.860362 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 11:47:10.861347 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 11:47:10.861347 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 11:47:10.861347 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:47:10.861347 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 249/502] 10.222.0.59 () {56 vars in 1108 bytes} [Tue Oct 11 11:47:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 11 11:47:10.867952 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9faf4fa-e676-409d-9fcc-7d3d71cb0431 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:10.989375 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9faf4fa-e676-409d-9fcc-7d3d71cb0431 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:11.037705 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ead1dcd-5b3d-4af0-ab55-05fbee7c174d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:11.038748 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ead1dcd-5b3d-4af0-ab55-05fbee7c174d None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:11.039250 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ead1dcd-5b3d-4af0-ab55-05fbee7c174d None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:11.039718 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ead1dcd-5b3d-4af0-ab55-05fbee7c174d None admin] PATH_INFO: `/v3/domains` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:11.040562 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ead1dcd-5b3d-4af0-ab55-05fbee7c174d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:11.041088 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ead1dcd-5b3d-4af0-ab55-05fbee7c174d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:11.042751 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ead1dcd-5b3d-4af0-ab55-05fbee7c174d None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:11.066115 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 254/503] 10.222.0.59 () {56 vars in 1109 bytes} [Tue Oct 11 11:47:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:47:11.075977 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f4edd5-152b-47e5-a434-dec128ab9a4f None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:11.231309 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f4edd5-152b-47e5-a434-dec128ab9a4f None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:11.243089 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:11.273436 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fba6045-92fa-43c0-b5ee-e3c5655b591d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:11.274098 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fba6045-92fa-43c0-b5ee-e3c5655b591d None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:11.274381 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fba6045-92fa-43c0-b5ee-e3c5655b591d None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:11.274671 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fba6045-92fa-43c0-b5ee-e3c5655b591d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:11.286452 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fba6045-92fa-43c0-b5ee-e3c5655b591d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:11.287988 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fba6045-92fa-43c0-b5ee-e3c5655b591d None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:11.303032 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application [None req-9fba6045-92fa-43c0-b5ee-e3c5655b591d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 11:47:11.303032 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:47:11.303032 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:47:11.303032 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:47:11.303032 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:47:11.303032 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:47:11.303032 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:47:11.303032 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:47:11.303032 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:47:11.303032 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:47:11.303032 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:47:11.303032 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:47:11.303032 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 11 11:47:11.303032 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 11 11:47:11.303032 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 11 11:47:11.304817 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 11 11:47:11.304817 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:47:11.304817 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:47:11.304817 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:47:11.304817 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:47:11.304817 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:47:11.304817 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:47:11.304817 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:47:11.304817 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application with Lock( Oct 11 11:47:11.304817 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:47:11.304817 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:47:11.304817 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:47:11.304817 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:47:11.304817 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:47:11.304817 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:47:11.304817 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:47:11.304817 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:47:11.306486 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 11 11:47:11.306486 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 11 11:47:11.306486 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 11 11:47:11.306486 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 11 11:47:11.306486 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 11 11:47:11.306486 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 11 11:47:11.306486 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 11:47:11.306486 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:47:11.306486 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 250/504] 10.222.0.59 () {56 vars in 1111 bytes} [Tue Oct 11 11:47:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:47:11.310179 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ead1dcd-5b3d-4af0-ab55-05fbee7c174d None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:11.325896 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f185c62-b2ae-47ea-9678-5b1610f91c28 None None] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:11.326283 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f185c62-b2ae-47ea-9678-5b1610f91c28 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:11.326452 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f185c62-b2ae-47ea-9678-5b1610f91c28 None None] PATH_INFO: `/` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:11.327856 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 251/505] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:47:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 11 11:47:11.333505 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71512b81-d532-40b4-8556-2729dca176d8 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:11.334053 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71512b81-d532-40b4-8556-2729dca176d8 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:11.334490 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71512b81-d532-40b4-8556-2729dca176d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:11.369168 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-71512b81-d532-40b4-8556-2729dca176d8 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:47:11.370416 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-71512b81-d532-40b4-8556-2729dca176d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:47:11.435234 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:11.491140 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 252/506] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:47:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 11 11:47:11.531376 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ead1dcd-5b3d-4af0-ab55-05fbee7c174d None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:11.574407 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7ac8e3-a8f4-43cf-a36e-1a87d2c5cefa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:11.575409 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7ac8e3-a8f4-43cf-a36e-1a87d2c5cefa None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:11.575912 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7ac8e3-a8f4-43cf-a36e-1a87d2c5cefa None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:11.576435 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7ac8e3-a8f4-43cf-a36e-1a87d2c5cefa None admin] PATH_INFO: `/v3/projects` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:11.577362 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7ac8e3-a8f4-43cf-a36e-1a87d2c5cefa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:11.577884 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7ac8e3-a8f4-43cf-a36e-1a87d2c5cefa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:11.579654 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7ac8e3-a8f4-43cf-a36e-1a87d2c5cefa None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:11.595451 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 255/507] 10.222.0.59 () {56 vars in 1148 bytes} [Tue Oct 11 11:47:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:47:11.602267 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71512b81-d532-40b4-8556-2729dca176d8 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:11.720878 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:11.727680 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71512b81-d532-40b4-8556-2729dca176d8 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:11.794607 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef43a1f-8850-4ebe-8379-d09a53f0d04e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:11.795588 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef43a1f-8850-4ebe-8379-d09a53f0d04e None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:11.796135 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef43a1f-8850-4ebe-8379-d09a53f0d04e None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:11.796612 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef43a1f-8850-4ebe-8379-d09a53f0d04e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:11.798063 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef43a1f-8850-4ebe-8379-d09a53f0d04e None admin] RBAC: Adding query filter params (role.id=f5a43d8f3fa24a87a7abe7492cf6f87f, scope.project.id=26ecf552cc9a4e5ca1d380509683dc4a, user.id=76e7fee1993a40d5ba4295e7daeb3e29) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:11.798686 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef43a1f-8850-4ebe-8379-d09a53f0d04e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:11.801575 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef43a1f-8850-4ebe-8379-d09a53f0d04e None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:11.813490 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 253/508] 10.222.0.59 () {56 vars in 1374 bytes} [Tue Oct 11 11:47:11 2022] GET /identity/v3/role_assignments?user.id=76e7fee1993a40d5ba4295e7daeb3e29&scope.project.id=26ecf552cc9a4e5ca1d380509683dc4a&role.id=f5a43d8f3fa24a87a7abe7492cf6f87f => generated 608 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:47:11.822937 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50ef603-48fe-4cdf-8a37-f4130d9bf5b0 None None] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:11.823787 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50ef603-48fe-4cdf-8a37-f4130d9bf5b0 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:11.824514 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50ef603-48fe-4cdf-8a37-f4130d9bf5b0 None None] PATH_INFO: `/` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:11.827786 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 256/509] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:47:11 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:47:11.837489 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafcba32-a52a-4f18-86e3-735d8ad6c172 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:11.837971 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafcba32-a52a-4f18-86e3-735d8ad6c172 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:11.838447 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafcba32-a52a-4f18-86e3-735d8ad6c172 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:11.881081 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-bafcba32-a52a-4f18-86e3-735d8ad6c172 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:47:11.883943 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-bafcba32-a52a-4f18-86e3-735d8ad6c172 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:47:11.958405 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50ef603-48fe-4cdf-8a37-f4130d9bf5b0 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:12.028192 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 254/510] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:47:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 11:47:12.038303 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87013cb2-9408-4521-b0d0-7db42b9c5249 None None] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:12.039289 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87013cb2-9408-4521-b0d0-7db42b9c5249 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:12.040040 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87013cb2-9408-4521-b0d0-7db42b9c5249 None None] PATH_INFO: `/` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:12.043372 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 255/511] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:47:12 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:47:12.052546 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a00010e-ea17-414d-a262-2dbf90004561 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:12.053439 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a00010e-ea17-414d-a262-2dbf90004561 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:12.054171 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a00010e-ea17-414d-a262-2dbf90004561 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:12.087064 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-3a00010e-ea17-414d-a262-2dbf90004561 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:47:12.090920 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-3a00010e-ea17-414d-a262-2dbf90004561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:47:12.170969 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50ef603-48fe-4cdf-8a37-f4130d9bf5b0 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:12.187190 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:12.228440 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3826fc-5d3e-4a9a-99ad-c4273bb3a7ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:12.228945 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3826fc-5d3e-4a9a-99ad-c4273bb3a7ba None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:12.229221 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3826fc-5d3e-4a9a-99ad-c4273bb3a7ba None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:12.229509 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3826fc-5d3e-4a9a-99ad-c4273bb3a7ba None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:12.244075 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3826fc-5d3e-4a9a-99ad-c4273bb3a7ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 11:47:12.248906 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application [None req-ed3826fc-5d3e-4a9a-99ad-c4273bb3a7ba None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 11 11:47:12.248906 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:47:12.248906 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:47:12.248906 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:47:12.248906 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:47:12.248906 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:47:12.248906 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:47:12.248906 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:47:12.248906 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:47:12.248906 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:47:12.248906 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:47:12.248906 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:47:12.248906 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 11:47:12.248906 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 11:47:12.248906 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 11:47:12.248906 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 11:47:12.250655 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 11:47:12.250655 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 11:47:12.250655 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 11:47:12.250655 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 11:47:12.250655 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:47:12.250655 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:47:12.250655 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:47:12.250655 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:47:12.250655 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:47:12.250655 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:47:12.250655 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:47:12.250655 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application with Lock( Oct 11 11:47:12.250655 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:47:12.250655 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:47:12.250655 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:47:12.250655 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:47:12.252282 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:47:12.252282 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:47:12.252282 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:47:12.252282 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:47:12.252282 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 11:47:12.252282 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 11:47:12.252282 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 11:47:12.252282 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 11:47:12.252282 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 11:47:12.252282 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 11:47:12.252282 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 11 11:47:12.252282 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:47:12.252282 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 257/512] 10.222.0.59 () {56 vars in 1096 bytes} [Tue Oct 11 11:47:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:47:12.256958 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 256/513] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:47:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 11 11:47:12.260221 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3826fc-5d3e-4a9a-99ad-c4273bb3a7ba None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:12.354104 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a00010e-ea17-414d-a262-2dbf90004561 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:12.397205 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:12.445454 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3826fc-5d3e-4a9a-99ad-c4273bb3a7ba None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:12.493000 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8236196-4d2b-42d7-8fa2-72a8df7a262f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:12.493891 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8236196-4d2b-42d7-8fa2-72a8df7a262f None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:12.495198 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8236196-4d2b-42d7-8fa2-72a8df7a262f None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:12.497229 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8236196-4d2b-42d7-8fa2-72a8df7a262f None admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:12.498139 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8236196-4d2b-42d7-8fa2-72a8df7a262f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:12.498620 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8236196-4d2b-42d7-8fa2-72a8df7a262f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:12.501507 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8236196-4d2b-42d7-8fa2-72a8df7a262f None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:12.518059 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 258/514] 10.222.0.59 () {56 vars in 1099 bytes} [Tue Oct 11 11:47:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:47:12.519577 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31d842b-e29d-4975-bbdd-3b096e10b619 None None] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:12.519869 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31d842b-e29d-4975-bbdd-3b096e10b619 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:12.520180 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31d842b-e29d-4975-bbdd-3b096e10b619 None None] PATH_INFO: `/` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:12.521548 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 259/515] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:47:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:47:12.522604 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31d842b-e29d-4975-bbdd-3b096e10b619 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:12.549923 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a00010e-ea17-414d-a262-2dbf90004561 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:12.565896 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:12.593336 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecdf870-187f-44e5-b4cf-4923a6f91bfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:12.594389 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecdf870-187f-44e5-b4cf-4923a6f91bfe None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:12.594730 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecdf870-187f-44e5-b4cf-4923a6f91bfe None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:12.595011 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecdf870-187f-44e5-b4cf-4923a6f91bfe None admin] PATH_INFO: `/v3/services/network` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:12.596049 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecdf870-187f-44e5-b4cf-4923a6f91bfe None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:12.597620 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecdf870-187f-44e5-b4cf-4923a6f91bfe None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:12.616537 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application [None req-9ecdf870-187f-44e5-b4cf-4923a6f91bfe None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 11 11:47:12.616537 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:47:12.616537 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 11 11:47:12.616537 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 11 11:47:12.616537 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 11 11:47:12.616537 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 11 11:47:12.616537 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 11 11:47:12.616537 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 11:47:12.616537 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 11 11:47:12.616537 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:47:12.616537 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 11 11:47:12.616537 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:47:12.616537 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:47:12.616537 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:47:12.616537 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:47:12.616537 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:47:12.616537 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:47:12.618323 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:47:12.618323 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:47:12.618323 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:47:12.618323 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:47:12.618323 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:47:12.618323 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:47:12.618323 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 11 11:47:12.618323 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 11 11:47:12.618323 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 11 11:47:12.618323 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 11 11:47:12.618323 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:47:12.618323 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:47:12.618323 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:47:12.618323 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:47:12.618323 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:47:12.618323 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:47:12.619445 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:47:12.619445 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application with Lock( Oct 11 11:47:12.619445 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:47:12.619445 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:47:12.619445 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:47:12.619445 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:47:12.619445 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:47:12.619445 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:47:12.619445 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:47:12.619445 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:47:12.619445 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 11 11:47:12.619445 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 11:47:12.619445 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 11 11:47:12.619445 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:47:12.619445 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 257/516] 10.222.0.59 () {56 vars in 1111 bytes} [Tue Oct 11 11:47:12 2022] GET /identity/v3/services/network => generated 88 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 11 11:47:12.620260 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72d6575-9e35-41e8-88d7-b9bc8386b2ac None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:12.620260 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72d6575-9e35-41e8-88d7-b9bc8386b2ac None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:12.620260 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72d6575-9e35-41e8-88d7-b9bc8386b2ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:12.643583 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-e72d6575-9e35-41e8-88d7-b9bc8386b2ac None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:47:12.644821 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-e72d6575-9e35-41e8-88d7-b9bc8386b2ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:47:12.654011 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31d842b-e29d-4975-bbdd-3b096e10b619 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:12.693256 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba904fe-bdc1-4bf6-b4be-7a4c627496a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:12.693941 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba904fe-bdc1-4bf6-b4be-7a4c627496a7 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:12.694227 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba904fe-bdc1-4bf6-b4be-7a4c627496a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:12.694467 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba904fe-bdc1-4bf6-b4be-7a4c627496a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:12.706653 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba904fe-bdc1-4bf6-b4be-7a4c627496a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:12.708380 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba904fe-bdc1-4bf6-b4be-7a4c627496a7 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:12.725485 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application [None req-aba904fe-bdc1-4bf6-b4be-7a4c627496a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 11:47:12.725485 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:47:12.725485 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:47:12.725485 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:47:12.725485 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:47:12.725485 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:47:12.725485 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:47:12.725485 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:47:12.725485 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:47:12.725485 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:47:12.725485 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:47:12.725485 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:47:12.725485 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 11:47:12.725485 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 11:47:12.725485 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 11:47:12.725485 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 11:47:12.728187 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:47:12.728187 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:47:12.728187 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:47:12.728187 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:47:12.728187 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:47:12.728187 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:47:12.728187 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:47:12.728187 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application with Lock( Oct 11 11:47:12.728187 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:47:12.728187 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:47:12.728187 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:47:12.728187 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:47:12.728187 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:47:12.728187 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:47:12.728187 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:47:12.728187 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:47:12.728187 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 11:47:12.730458 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 11:47:12.730458 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 11:47:12.730458 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:47:12.730458 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 260/517] 10.222.0.59 () {56 vars in 1108 bytes} [Tue Oct 11 11:47:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:47:12.730458 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba904fe-bdc1-4bf6-b4be-7a4c627496a7 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:12.732959 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 258/518] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:47:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 11:47:12.733890 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72d6575-9e35-41e8-88d7-b9bc8386b2ac None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:12.773784 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:12.879513 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:12.916876 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba904fe-bdc1-4bf6-b4be-7a4c627496a7 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:12.936991 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72d6575-9e35-41e8-88d7-b9bc8386b2ac None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:13.008419 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d480319-9848-49a3-89ed-4a3a3f034121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:13.009491 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d480319-9848-49a3-89ed-4a3a3f034121 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:13.009967 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d480319-9848-49a3-89ed-4a3a3f034121 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:13.010445 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d480319-9848-49a3-89ed-4a3a3f034121 None admin] PATH_INFO: `/v3/services` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:13.011736 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d480319-9848-49a3-89ed-4a3a3f034121 None admin] RBAC: Adding query filter params (name=network) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:13.012219 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d480319-9848-49a3-89ed-4a3a3f034121 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:13.012369 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacbaab1-3706-472d-a554-6a73adeb3315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:13.013052 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacbaab1-3706-472d-a554-6a73adeb3315 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:13.013324 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacbaab1-3706-472d-a554-6a73adeb3315 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:13.013600 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacbaab1-3706-472d-a554-6a73adeb3315 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:13.014914 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d480319-9848-49a3-89ed-4a3a3f034121 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:13.032767 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 261/519] 10.222.0.59 () {56 vars in 1112 bytes} [Tue Oct 11 11:47:12 2022] GET /identity/v3/services?name=network => generated 124 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:47:13.033102 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacbaab1-3706-472d-a554-6a73adeb3315 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 11:47:13.033237 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d480319-9848-49a3-89ed-4a3a3f034121 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:13.033751 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application [None req-dacbaab1-3706-472d-a554-6a73adeb3315 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 11 11:47:13.033751 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:47:13.033751 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:47:13.033751 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:47:13.033751 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:47:13.033751 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:47:13.033751 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:47:13.033751 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:47:13.033751 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:47:13.033751 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:47:13.033751 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:47:13.033751 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:47:13.033751 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 11:47:13.033751 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 11:47:13.033751 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 11:47:13.033751 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 11:47:13.035614 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 11:47:13.035614 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 11:47:13.035614 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 11:47:13.035614 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 11:47:13.035614 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:47:13.035614 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:47:13.035614 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:47:13.035614 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:47:13.035614 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:47:13.035614 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:47:13.035614 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:47:13.035614 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application with Lock( Oct 11 11:47:13.035614 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:47:13.035614 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:47:13.035614 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:47:13.035614 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:47:13.036454 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:47:13.036454 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:47:13.036454 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:47:13.036454 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:47:13.036454 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 11:47:13.036454 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 11:47:13.036454 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 11:47:13.036454 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 11:47:13.036454 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 11:47:13.036454 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 11:47:13.036454 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 11 11:47:13.036454 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:47:13.036454 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 259/520] 10.222.0.59 () {56 vars in 1102 bytes} [Tue Oct 11 11:47:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:47:13.036454 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacbaab1-3706-472d-a554-6a73adeb3315 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:13.179994 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:13.197375 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d480319-9848-49a3-89ed-4a3a3f034121 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:13.217878 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacbaab1-3706-472d-a554-6a73adeb3315 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:13.264982 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93097542-c771-42f7-b9e5-cd29ca1e3d6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:13.264982 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93097542-c771-42f7-b9e5-cd29ca1e3d6a None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:13.264982 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93097542-c771-42f7-b9e5-cd29ca1e3d6a None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:13.264982 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93097542-c771-42f7-b9e5-cd29ca1e3d6a None admin] PATH_INFO: `/v3/domains` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:13.265855 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93097542-c771-42f7-b9e5-cd29ca1e3d6a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:13.265855 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93097542-c771-42f7-b9e5-cd29ca1e3d6a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:13.265855 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93097542-c771-42f7-b9e5-cd29ca1e3d6a None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:13.288420 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 262/521] 10.222.0.59 () {56 vars in 1109 bytes} [Tue Oct 11 11:47:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:47:13.288420 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93097542-c771-42f7-b9e5-cd29ca1e3d6a None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:13.293305 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1ee994-7ee7-4c1b-bc7f-b042a075c2dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:13.294379 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1ee994-7ee7-4c1b-bc7f-b042a075c2dd None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:13.294867 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1ee994-7ee7-4c1b-bc7f-b042a075c2dd None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:13.295283 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1ee994-7ee7-4c1b-bc7f-b042a075c2dd None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:13.296161 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1ee994-7ee7-4c1b-bc7f-b042a075c2dd None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:13.298621 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1ee994-7ee7-4c1b-bc7f-b042a075c2dd None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:13.305878 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:13.314079 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application [None req-6a1ee994-7ee7-4c1b-bc7f-b042a075c2dd None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 11 11:47:13.314079 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:47:13.314079 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 11 11:47:13.314079 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 11 11:47:13.314079 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 11 11:47:13.314079 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 11 11:47:13.314079 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 11 11:47:13.314079 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 11:47:13.314079 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 11 11:47:13.314079 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:47:13.314079 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 11 11:47:13.314079 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:47:13.314079 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:47:13.314079 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:47:13.314079 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:47:13.314079 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:47:13.314079 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:47:13.316028 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:47:13.316028 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:47:13.316028 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:47:13.316028 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:47:13.316028 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:47:13.316028 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:47:13.316028 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 11 11:47:13.316028 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 11 11:47:13.316028 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 11 11:47:13.316028 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 11 11:47:13.316028 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:47:13.316028 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:47:13.316028 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:47:13.316028 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:47:13.316028 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:47:13.316028 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:47:13.317573 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:47:13.317573 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application with Lock( Oct 11 11:47:13.317573 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:47:13.317573 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:47:13.317573 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:47:13.317573 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:47:13.317573 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:47:13.317573 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:47:13.317573 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:47:13.317573 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:47:13.317573 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 11 11:47:13.317573 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 11:47:13.317573 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 11 11:47:13.317573 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:47:13.317573 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 260/522] 10.222.0.59 () {56 vars in 1129 bytes} [Tue Oct 11 11:47:13 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:47:13.321070 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1ee994-7ee7-4c1b-bc7f-b042a075c2dd None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:13.467149 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93097542-c771-42f7-b9e5-cd29ca1e3d6a None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:13.484827 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1ee994-7ee7-4c1b-bc7f-b042a075c2dd None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:13.528682 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11a0f4d-a2c5-45b4-aaf9-72df29553b98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:13.529686 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11a0f4d-a2c5-45b4-aaf9-72df29553b98 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:13.530140 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11a0f4d-a2c5-45b4-aaf9-72df29553b98 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:13.530618 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11a0f4d-a2c5-45b4-aaf9-72df29553b98 None admin] PATH_INFO: `/v3/services` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:13.531771 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11a0f4d-a2c5-45b4-aaf9-72df29553b98 None admin] RBAC: Adding query filter params (type=network) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:13.532394 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11a0f4d-a2c5-45b4-aaf9-72df29553b98 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:13.533636 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89872e09-f1a4-4d30-bbe7-33bf6e1f9b79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:13.534263 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89872e09-f1a4-4d30-bbe7-33bf6e1f9b79 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:13.534920 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89872e09-f1a4-4d30-bbe7-33bf6e1f9b79 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:13.535224 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89872e09-f1a4-4d30-bbe7-33bf6e1f9b79 None admin] PATH_INFO: `/v3/roles` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:13.535385 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11a0f4d-a2c5-45b4-aaf9-72df29553b98 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:13.535878 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89872e09-f1a4-4d30-bbe7-33bf6e1f9b79 None admin] RBAC: Adding query filter params (name=service) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:13.536173 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89872e09-f1a4-4d30-bbe7-33bf6e1f9b79 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:13.537565 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89872e09-f1a4-4d30-bbe7-33bf6e1f9b79 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:13.546185 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 263/523] 10.222.0.59 () {56 vars in 1112 bytes} [Tue Oct 11 11:47:13 2022] GET /identity/v3/services?type=network => generated 124 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:47:13.548074 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11a0f4d-a2c5-45b4-aaf9-72df29553b98 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:13.562886 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 261/524] 10.222.0.59 () {56 vars in 1103 bytes} [Tue Oct 11 11:47:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:47:13.564930 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89872e09-f1a4-4d30-bbe7-33bf6e1f9b79 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:13.575151 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:13.719104 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:13.752350 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11a0f4d-a2c5-45b4-aaf9-72df29553b98 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:13.815841 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89872e09-f1a4-4d30-bbe7-33bf6e1f9b79 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:13.826981 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb34cd6-da19-4d62-b2fe-10e264feb989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:13.828141 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb34cd6-da19-4d62-b2fe-10e264feb989 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:13.828650 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb34cd6-da19-4d62-b2fe-10e264feb989 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:13.829130 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb34cd6-da19-4d62-b2fe-10e264feb989 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:13.848795 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb34cd6-da19-4d62-b2fe-10e264feb989 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:13.850999 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb34cd6-da19-4d62-b2fe-10e264feb989 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:13.869020 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application [None req-8fb34cd6-da19-4d62-b2fe-10e264feb989 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 11 11:47:13.869020 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:47:13.869020 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:47:13.869020 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:47:13.869020 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:47:13.869020 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:47:13.869020 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:47:13.869020 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:47:13.869020 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:47:13.869020 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:47:13.869020 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:47:13.869020 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:47:13.869020 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 11 11:47:13.869020 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 11 11:47:13.869020 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 11 11:47:13.869020 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 11 11:47:13.870527 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:47:13.870527 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:47:13.870527 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 11 11:47:13.870527 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 11:47:13.870527 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 11 11:47:13.870527 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 11:47:13.870527 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:47:13.870527 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:47:13.870527 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:47:13.870527 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:47:13.870527 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:47:13.870527 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application with Lock( Oct 11 11:47:13.870527 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:47:13.870527 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:47:13.870527 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:47:13.870527 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:47:13.870527 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:47:13.870527 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:47:13.872267 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:47:13.872267 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:47:13.872267 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 11 11:47:13.872267 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 11 11:47:13.872267 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 11 11:47:13.872267 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 11 11:47:13.872267 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 11 11:47:13.872267 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 11 11:47:13.872267 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 11 11:47:13.872267 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:47:13.873234 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 264/525] 10.222.0.59 () {56 vars in 1093 bytes} [Tue Oct 11 11:47:13 2022] GET /identity/v3/users/nova => generated 82 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:47:13.874382 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb34cd6-da19-4d62-b2fe-10e264feb989 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:13.892602 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520a11df-1da8-4983-9db9-edc643c8172a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:13.893604 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520a11df-1da8-4983-9db9-edc643c8172a None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:13.895390 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520a11df-1da8-4983-9db9-edc643c8172a None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:13.895390 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520a11df-1da8-4983-9db9-edc643c8172a None admin] PATH_INFO: `/v3/services` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:13.896423 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520a11df-1da8-4983-9db9-edc643c8172a None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:13.896992 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520a11df-1da8-4983-9db9-edc643c8172a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:13.899683 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520a11df-1da8-4983-9db9-edc643c8172a None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:13.908506 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 262/526] 10.222.0.59 () {56 vars in 1124 bytes} [Tue Oct 11 11:47:13 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:47:13.910262 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520a11df-1da8-4983-9db9-edc643c8172a None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:14.072427 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:14.097856 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb34cd6-da19-4d62-b2fe-10e264feb989 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:14.153521 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520a11df-1da8-4983-9db9-edc643c8172a None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:14.158525 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:14.169636 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30daed2-2788-4668-914f-3618cf40379e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:14.170648 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30daed2-2788-4668-914f-3618cf40379e None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:14.171101 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30daed2-2788-4668-914f-3618cf40379e None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:14.171554 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30daed2-2788-4668-914f-3618cf40379e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:14.193032 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30daed2-2788-4668-914f-3618cf40379e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:14.195618 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30daed2-2788-4668-914f-3618cf40379e None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:14.222703 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application [None req-c30daed2-2788-4668-914f-3618cf40379e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 11:47:14.222703 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:47:14.222703 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:47:14.222703 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:47:14.222703 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:47:14.222703 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:47:14.222703 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:47:14.222703 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:47:14.222703 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:47:14.222703 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:47:14.222703 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:47:14.222703 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:47:14.222703 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 11:47:14.222703 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 11:47:14.222703 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 11:47:14.222703 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 11:47:14.224752 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:47:14.224752 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:47:14.224752 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:47:14.224752 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:47:14.224752 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:47:14.224752 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:47:14.224752 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:47:14.224752 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application with Lock( Oct 11 11:47:14.224752 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:47:14.224752 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:47:14.224752 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:47:14.224752 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:47:14.224752 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:47:14.224752 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:47:14.224752 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:47:14.224752 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:47:14.224752 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 11:47:14.226592 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 11:47:14.226592 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 11:47:14.226592 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:47:14.226592 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 265/527] 10.222.0.59 () {56 vars in 1108 bytes} [Tue Oct 11 11:47:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 351 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:47:14.226592 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30daed2-2788-4668-914f-3618cf40379e None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:14.238279 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c39dfa7-76ca-48f6-9d75-6c660eab7e5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:14.240374 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c39dfa7-76ca-48f6-9d75-6c660eab7e5c None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:14.243150 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c39dfa7-76ca-48f6-9d75-6c660eab7e5c None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:14.243749 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c39dfa7-76ca-48f6-9d75-6c660eab7e5c None admin] PATH_INFO: `/v3/users` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:14.245241 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c39dfa7-76ca-48f6-9d75-6c660eab7e5c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:14.245906 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c39dfa7-76ca-48f6-9d75-6c660eab7e5c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:14.250377 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c39dfa7-76ca-48f6-9d75-6c660eab7e5c None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:14.298149 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 263/528] 10.222.0.59 () {56 vars in 1133 bytes} [Tue Oct 11 11:47:13 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 389 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:47:14.300119 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c39dfa7-76ca-48f6-9d75-6c660eab7e5c None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:14.446653 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30daed2-2788-4668-914f-3618cf40379e None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:14.491348 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510dd9aa-9190-4dbe-b461-c9aa1d9ac73f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:14.492394 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510dd9aa-9190-4dbe-b461-c9aa1d9ac73f None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:14.492765 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510dd9aa-9190-4dbe-b461-c9aa1d9ac73f None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:14.493185 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510dd9aa-9190-4dbe-b461-c9aa1d9ac73f None admin] PATH_INFO: `/v3/services` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:14.494124 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510dd9aa-9190-4dbe-b461-c9aa1d9ac73f None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:14.494473 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510dd9aa-9190-4dbe-b461-c9aa1d9ac73f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:14.496025 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510dd9aa-9190-4dbe-b461-c9aa1d9ac73f None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:14.502097 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 266/529] 10.222.0.59 () {56 vars in 1124 bytes} [Tue Oct 11 11:47:14 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:47:14.503564 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510dd9aa-9190-4dbe-b461-c9aa1d9ac73f None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:14.525921 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c39dfa7-76ca-48f6-9d75-6c660eab7e5c None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:14.583762 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:14.598519 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:14.601305 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167297b1-c131-49b4-87bc-85b373e64f4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:14.602818 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167297b1-c131-49b4-87bc-85b373e64f4a None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:14.603583 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167297b1-c131-49b4-87bc-85b373e64f4a None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:14.604473 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167297b1-c131-49b4-87bc-85b373e64f4a None admin] PATH_INFO: `/v3/domains` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:14.606265 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167297b1-c131-49b4-87bc-85b373e64f4a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:14.607099 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167297b1-c131-49b4-87bc-85b373e64f4a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:14.609974 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167297b1-c131-49b4-87bc-85b373e64f4a None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:14.629720 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 264/530] 10.222.0.59 () {56 vars in 1109 bytes} [Tue Oct 11 11:47:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:47:14.632228 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef39c6dd-ba26-49bc-9203-7a70cabdd405 None None] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:14.633083 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef39c6dd-ba26-49bc-9203-7a70cabdd405 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:14.633806 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef39c6dd-ba26-49bc-9203-7a70cabdd405 None None] PATH_INFO: `/` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:14.637079 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 265/531] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:47:14 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:47:14.639713 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef39c6dd-ba26-49bc-9203-7a70cabdd405 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:14.724891 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510dd9aa-9190-4dbe-b461-c9aa1d9ac73f None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:14.805903 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52936a6-0a83-4a12-908d-f9b7f775b91f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:14.807469 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52936a6-0a83-4a12-908d-f9b7f775b91f None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:14.808290 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52936a6-0a83-4a12-908d-f9b7f775b91f None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:14.809030 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52936a6-0a83-4a12-908d-f9b7f775b91f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:14.829059 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52936a6-0a83-4a12-908d-f9b7f775b91f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:14.832251 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52936a6-0a83-4a12-908d-f9b7f775b91f None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:14.856357 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application [None req-f52936a6-0a83-4a12-908d-f9b7f775b91f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 11:47:14.856357 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:47:14.856357 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:47:14.856357 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:47:14.856357 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:47:14.856357 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:47:14.856357 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:47:14.856357 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:47:14.856357 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:47:14.856357 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:47:14.856357 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:47:14.856357 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:47:14.856357 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 11:47:14.856357 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 11:47:14.856357 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 11:47:14.856357 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 11:47:14.858062 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:47:14.858062 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:47:14.858062 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:47:14.858062 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:47:14.858062 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:47:14.858062 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:47:14.858062 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:47:14.858062 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application with Lock( Oct 11 11:47:14.858062 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:47:14.858062 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:47:14.858062 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:47:14.858062 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:47:14.858062 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:47:14.858062 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:47:14.858062 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:47:14.858062 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:47:14.858062 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 11:47:14.859695 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 11:47:14.859695 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 11:47:14.859695 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:47:14.862829 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 267/532] 10.222.0.59 () {56 vars in 1108 bytes} [Tue Oct 11 11:47:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 360 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:47:14.870654 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef39c6dd-ba26-49bc-9203-7a70cabdd405 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:14.871467 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e9f531-984f-4883-baee-a4f3feb88857 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:14.871903 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e9f531-984f-4883-baee-a4f3feb88857 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:14.872478 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e9f531-984f-4883-baee-a4f3feb88857 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:14.905157 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-b7e9f531-984f-4883-baee-a4f3feb88857 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:47:14.907415 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-b7e9f531-984f-4883-baee-a4f3feb88857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:47:14.943450 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa65ba0f-1f5b-4e81-8384-cf65678256d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:14.944941 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa65ba0f-1f5b-4e81-8384-cf65678256d2 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:14.945708 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa65ba0f-1f5b-4e81-8384-cf65678256d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:14.946400 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa65ba0f-1f5b-4e81-8384-cf65678256d2 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:14.964520 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa65ba0f-1f5b-4e81-8384-cf65678256d2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:14.967230 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa65ba0f-1f5b-4e81-8384-cf65678256d2 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:14.997703 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application [None req-fa65ba0f-1f5b-4e81-8384-cf65678256d2 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 11 11:47:14.997703 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:47:14.997703 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:47:14.997703 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:47:14.997703 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:47:14.997703 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:47:14.997703 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:47:14.997703 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:47:14.997703 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:47:14.997703 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:47:14.997703 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:47:14.997703 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:47:14.997703 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 11 11:47:14.997703 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 11 11:47:14.997703 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 11 11:47:14.997703 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 11 11:47:14.999613 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:47:14.999613 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:47:14.999613 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 11 11:47:14.999613 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 11:47:14.999613 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 11 11:47:14.999613 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 11:47:14.999613 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:47:14.999613 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:47:14.999613 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:47:14.999613 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:47:14.999613 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:47:14.999613 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application with Lock( Oct 11 11:47:14.999613 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:47:14.999613 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:47:14.999613 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:47:14.999613 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:47:14.999613 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:47:14.999613 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:47:15.002390 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:47:15.002390 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:47:15.002390 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 11 11:47:15.002390 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 11 11:47:15.002390 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 11 11:47:15.002390 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 11 11:47:15.002390 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 11 11:47:15.002390 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 11 11:47:15.002390 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 11 11:47:15.002390 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:47:15.002390 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 266/533] 10.222.0.59 () {56 vars in 1117 bytes} [Tue Oct 11 11:47:14 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 361 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:47:15.002390 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa65ba0f-1f5b-4e81-8384-cf65678256d2 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:15.047597 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 268/534] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:47:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 11:47:15.054525 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:15.055461 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e9f531-984f-4883-baee-a4f3feb88857 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:15.120866 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa65ba0f-1f5b-4e81-8384-cf65678256d2 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:15.149094 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:15.162684 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e374bb-6ed6-42e9-92c7-ec5b188917b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:15.163602 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e374bb-6ed6-42e9-92c7-ec5b188917b6 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:15.164030 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e374bb-6ed6-42e9-92c7-ec5b188917b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:15.164439 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e374bb-6ed6-42e9-92c7-ec5b188917b6 None admin] PATH_INFO: `/v3/domains` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:15.165236 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e374bb-6ed6-42e9-92c7-ec5b188917b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:15.165682 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e374bb-6ed6-42e9-92c7-ec5b188917b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:15.167433 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e374bb-6ed6-42e9-92c7-ec5b188917b6 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:15.181618 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 267/535] 10.222.0.59 () {56 vars in 1109 bytes} [Tue Oct 11 11:47:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:47:15.187406 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e374bb-6ed6-42e9-92c7-ec5b188917b6 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:15.295729 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e374bb-6ed6-42e9-92c7-ec5b188917b6 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:15.304518 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:15.338311 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d1cb66-3404-4c1c-971b-1dfcd17394c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:15.338984 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d1cb66-3404-4c1c-971b-1dfcd17394c3 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:15.339259 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d1cb66-3404-4c1c-971b-1dfcd17394c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:15.339529 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d1cb66-3404-4c1c-971b-1dfcd17394c3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:15.354074 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d1cb66-3404-4c1c-971b-1dfcd17394c3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:15.356423 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d1cb66-3404-4c1c-971b-1dfcd17394c3 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:15.369431 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e9f531-984f-4883-baee-a4f3feb88857 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:15.384199 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application [None req-03d1cb66-3404-4c1c-971b-1dfcd17394c3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 11:47:15.384199 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:47:15.384199 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:47:15.384199 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:47:15.384199 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:47:15.384199 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:47:15.384199 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:47:15.384199 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:47:15.384199 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:47:15.384199 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:47:15.384199 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:47:15.384199 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:47:15.384199 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 11 11:47:15.384199 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 11 11:47:15.384199 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 11 11:47:15.385977 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 11 11:47:15.385977 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:47:15.385977 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:47:15.385977 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:47:15.385977 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:47:15.385977 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:47:15.385977 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:47:15.385977 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:47:15.385977 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application with Lock( Oct 11 11:47:15.385977 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:47:15.385977 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:47:15.385977 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:47:15.385977 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:47:15.385977 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:47:15.385977 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:47:15.385977 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:47:15.385977 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:47:15.386905 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 11 11:47:15.386905 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 11 11:47:15.386905 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 11 11:47:15.386905 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 11 11:47:15.386905 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 11 11:47:15.386905 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 11 11:47:15.386905 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 11:47:15.386905 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:47:15.386905 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 268/536] 10.222.0.59 () {56 vars in 1111 bytes} [Tue Oct 11 11:47:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:47:15.391655 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d1cb66-3404-4c1c-971b-1dfcd17394c3 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:15.445823 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f8336a-6a99-43a7-a981-9e8571d6365b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:15.446451 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f8336a-6a99-43a7-a981-9e8571d6365b None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:15.446751 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f8336a-6a99-43a7-a981-9e8571d6365b None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:15.447019 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f8336a-6a99-43a7-a981-9e8571d6365b None admin] PATH_INFO: `/v3/users` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:15.447760 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f8336a-6a99-43a7-a981-9e8571d6365b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:15.448060 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f8336a-6a99-43a7-a981-9e8571d6365b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:15.450965 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f8336a-6a99-43a7-a981-9e8571d6365b None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:15.476094 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 269/537] 10.222.0.59 () {56 vars in 1149 bytes} [Tue Oct 11 11:47:15 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 428 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:47:15.483988 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f8336a-6a99-43a7-a981-9e8571d6365b None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:15.603844 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d1cb66-3404-4c1c-971b-1dfcd17394c3 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:15.686288 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:15.688100 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c784765a-aff5-41cc-a64d-576fd20edb4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:15.688843 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c784765a-aff5-41cc-a64d-576fd20edb4e None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:15.689311 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c784765a-aff5-41cc-a64d-576fd20edb4e None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:15.689581 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c784765a-aff5-41cc-a64d-576fd20edb4e None admin] PATH_INFO: `/v3/projects` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:15.690531 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c784765a-aff5-41cc-a64d-576fd20edb4e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:15.690935 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c784765a-aff5-41cc-a64d-576fd20edb4e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:15.692368 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c784765a-aff5-41cc-a64d-576fd20edb4e None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:15.709939 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f8336a-6a99-43a7-a981-9e8571d6365b None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:15.724565 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 269/538] 10.222.0.59 () {56 vars in 1148 bytes} [Tue Oct 11 11:47:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:47:15.726263 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c784765a-aff5-41cc-a64d-576fd20edb4e None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:15.757547 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb923447-4512-4799-b85f-7b30adaa8086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:15.758505 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb923447-4512-4799-b85f-7b30adaa8086 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:15.758967 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb923447-4512-4799-b85f-7b30adaa8086 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:15.759247 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb923447-4512-4799-b85f-7b30adaa8086 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:15.772604 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb923447-4512-4799-b85f-7b30adaa8086 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:15.774215 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb923447-4512-4799-b85f-7b30adaa8086 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:15.785101 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:15.794877 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application [None req-cb923447-4512-4799-b85f-7b30adaa8086 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 11:47:15.794877 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:47:15.794877 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:47:15.794877 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:47:15.794877 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:47:15.794877 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:47:15.794877 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:47:15.794877 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:47:15.794877 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:47:15.794877 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:47:15.794877 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:47:15.794877 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:47:15.794877 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 11:47:15.794877 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 11:47:15.794877 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 11:47:15.794877 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 11:47:15.795906 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:47:15.795906 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:47:15.795906 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:47:15.795906 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:47:15.795906 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:47:15.795906 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:47:15.795906 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:47:15.795906 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application with Lock( Oct 11 11:47:15.795906 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:47:15.795906 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:47:15.795906 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:47:15.795906 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:47:15.795906 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:47:15.795906 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:47:15.795906 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:47:15.795906 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:47:15.795906 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 11:47:15.796781 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 11:47:15.796781 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 11:47:15.796781 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:47:15.796781 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 270/539] 10.222.0.59 () {56 vars in 1108 bytes} [Tue Oct 11 11:47:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:47:15.797442 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb923447-4512-4799-b85f-7b30adaa8086 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:15.938234 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb923447-4512-4799-b85f-7b30adaa8086 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:15.949454 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c784765a-aff5-41cc-a64d-576fd20edb4e None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:15.984588 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5bd034-f760-4898-93d5-d19d17739c37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:15.985237 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5bd034-f760-4898-93d5-d19d17739c37 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:15.985515 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5bd034-f760-4898-93d5-d19d17739c37 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:15.985781 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5bd034-f760-4898-93d5-d19d17739c37 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:15.997454 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5bd034-f760-4898-93d5-d19d17739c37 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 11:47:16.003833 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application [None req-4f5bd034-f760-4898-93d5-d19d17739c37 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 11 11:47:16.003833 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:47:16.003833 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:47:16.003833 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:47:16.003833 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:47:16.003833 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:47:16.003833 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:47:16.003833 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:47:16.003833 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:47:16.003833 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:47:16.003833 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:47:16.003833 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:47:16.003833 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 11:47:16.003833 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 11:47:16.003833 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 11:47:16.003833 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 11:47:16.005630 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 11:47:16.005630 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 11:47:16.005630 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 11:47:16.005630 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 11:47:16.005630 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:47:16.005630 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:47:16.005630 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:47:16.005630 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:47:16.005630 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:47:16.005630 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:47:16.005630 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:47:16.005630 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application with Lock( Oct 11 11:47:16.005630 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:47:16.005630 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:47:16.005630 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:47:16.005630 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:47:16.007277 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:47:16.007277 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:47:16.007277 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:47:16.007277 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:47:16.007277 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 11:47:16.007277 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 11:47:16.007277 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 11:47:16.007277 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 11:47:16.007277 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 11:47:16.007277 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 11:47:16.007277 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 11 11:47:16.007277 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:47:16.007277 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 270/540] 10.222.0.59 () {56 vars in 1096 bytes} [Tue Oct 11 11:47:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:47:16.007277 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5bd034-f760-4898-93d5-d19d17739c37 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:16.019215 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a5c799-ef6b-4cfb-a560-865647953c14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:16.020695 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a5c799-ef6b-4cfb-a560-865647953c14 None admin] REQUEST_METHOD: `PUT` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:16.021196 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a5c799-ef6b-4cfb-a560-865647953c14 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:16.021684 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a5c799-ef6b-4cfb-a560-865647953c14 None admin] PATH_INFO: `/v3/projects/26ecf552cc9a4e5ca1d380509683dc4a/users/bad5e1cb306b46818096940acc3810b2/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:16.025469 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a5c799-ef6b-4cfb-a560-865647953c14 None admin] RBAC: Authorizing `identity:create_grant(project_id=26ecf552cc9a4e5ca1d380509683dc4a, user_id=bad5e1cb306b46818096940acc3810b2, role_id=5cf1f4bd2fef46ab9c6eff56d52f9f69)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:16.028636 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a5c799-ef6b-4cfb-a560-865647953c14 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:16.048406 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:16.070102 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 271/541] 10.222.0.59 () {58 vars in 1439 bytes} [Tue Oct 11 11:47:15 2022] PUT /identity/v3/projects/26ecf552cc9a4e5ca1d380509683dc4a/users/bad5e1cb306b46818096940acc3810b2/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 11:47:16.070102 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a5c799-ef6b-4cfb-a560-865647953c14 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:16.165563 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:16.204056 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5bd034-f760-4898-93d5-d19d17739c37 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:16.287364 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f28d0c-f331-4dbe-805b-2515210a8dac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:16.287364 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f28d0c-f331-4dbe-805b-2515210a8dac None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:16.289259 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f28d0c-f331-4dbe-805b-2515210a8dac None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:16.290119 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f28d0c-f331-4dbe-805b-2515210a8dac None admin] PATH_INFO: `/v3/domains` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:16.290838 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f28d0c-f331-4dbe-805b-2515210a8dac None admin] RBAC: Adding query filter params (name=Default) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:16.291144 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f28d0c-f331-4dbe-805b-2515210a8dac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:16.294527 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f28d0c-f331-4dbe-805b-2515210a8dac None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:16.318166 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 271/542] 10.222.0.59 () {56 vars in 1109 bytes} [Tue Oct 11 11:47:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:47:16.322633 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f28d0c-f331-4dbe-805b-2515210a8dac None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:16.349926 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a5c799-ef6b-4cfb-a560-865647953c14 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:16.423144 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2a88a4-3130-46a3-8097-26bde2d4ce22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:16.423823 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2a88a4-3130-46a3-8097-26bde2d4ce22 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:16.424103 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2a88a4-3130-46a3-8097-26bde2d4ce22 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:16.424376 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2a88a4-3130-46a3-8097-26bde2d4ce22 None admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:16.425052 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2a88a4-3130-46a3-8097-26bde2d4ce22 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:16.425365 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2a88a4-3130-46a3-8097-26bde2d4ce22 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:16.426795 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2a88a4-3130-46a3-8097-26bde2d4ce22 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:16.436526 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 272/543] 10.222.0.59 () {56 vars in 1099 bytes} [Tue Oct 11 11:47:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 371 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:47:16.442975 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2a88a4-3130-46a3-8097-26bde2d4ce22 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:16.457628 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f28d0c-f331-4dbe-805b-2515210a8dac None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:16.509141 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c71fac-3900-4a49-aaea-79990c8f475c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:16.510451 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c71fac-3900-4a49-aaea-79990c8f475c None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:16.511114 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c71fac-3900-4a49-aaea-79990c8f475c None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:16.511594 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c71fac-3900-4a49-aaea-79990c8f475c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:16.522301 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:16.539513 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c71fac-3900-4a49-aaea-79990c8f475c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:16.542267 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c71fac-3900-4a49-aaea-79990c8f475c None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:16.569479 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2a88a4-3130-46a3-8097-26bde2d4ce22 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:16.587832 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application [None req-30c71fac-3900-4a49-aaea-79990c8f475c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 11:47:16.587832 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:47:16.587832 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:47:16.587832 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:47:16.587832 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:47:16.587832 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:47:16.587832 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:47:16.587832 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:47:16.587832 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:47:16.587832 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:47:16.587832 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:47:16.587832 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:47:16.587832 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 11 11:47:16.587832 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 11 11:47:16.587832 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 11 11:47:16.588835 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 11 11:47:16.588835 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:47:16.588835 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:47:16.588835 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:47:16.588835 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:47:16.588835 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:47:16.588835 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:47:16.588835 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:47:16.588835 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application with Lock( Oct 11 11:47:16.588835 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:47:16.588835 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:47:16.588835 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:47:16.588835 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:47:16.588835 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:47:16.588835 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:47:16.588835 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:47:16.588835 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:47:16.589700 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 11 11:47:16.589700 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 11 11:47:16.589700 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 11 11:47:16.589700 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 11 11:47:16.589700 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 11 11:47:16.589700 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 11 11:47:16.589700 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 11:47:16.589700 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:47:16.592281 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 272/544] 10.222.0.59 () {56 vars in 1111 bytes} [Tue Oct 11 11:47:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 11 11:47:16.598619 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c71fac-3900-4a49-aaea-79990c8f475c None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:16.617417 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73600e9d-85d5-4e68-b856-759e1432aaf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:16.618294 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73600e9d-85d5-4e68-b856-759e1432aaf1 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:16.618728 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73600e9d-85d5-4e68-b856-759e1432aaf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:16.619098 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73600e9d-85d5-4e68-b856-759e1432aaf1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:16.627215 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:16.640739 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73600e9d-85d5-4e68-b856-759e1432aaf1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:16.643647 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73600e9d-85d5-4e68-b856-759e1432aaf1 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:16.670704 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application [None req-73600e9d-85d5-4e68-b856-759e1432aaf1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 11:47:16.670704 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:47:16.670704 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:47:16.670704 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:47:16.670704 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:47:16.670704 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:47:16.670704 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:47:16.670704 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:47:16.670704 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:47:16.670704 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:47:16.670704 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:47:16.670704 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:47:16.670704 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 11:47:16.670704 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 11:47:16.670704 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 11:47:16.670704 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 11:47:16.671915 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:47:16.671915 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:47:16.671915 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:47:16.671915 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:47:16.671915 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:47:16.671915 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:47:16.671915 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:47:16.671915 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application with Lock( Oct 11 11:47:16.671915 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:47:16.671915 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:47:16.671915 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:47:16.671915 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:47:16.671915 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:47:16.671915 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:47:16.671915 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:47:16.671915 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:47:16.671915 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 11:47:16.672990 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 11:47:16.672990 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 11:47:16.672990 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:47:16.686172 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 273/545] 10.222.0.59 () {56 vars in 1108 bytes} [Tue Oct 11 11:47:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 11 11:47:16.695640 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73600e9d-85d5-4e68-b856-759e1432aaf1 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:16.833056 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c71fac-3900-4a49-aaea-79990c8f475c None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:16.904868 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73600e9d-85d5-4e68-b856-759e1432aaf1 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:16.908763 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4a48a2-8b75-45e2-86ef-4d1a5fa4ed2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:16.909747 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4a48a2-8b75-45e2-86ef-4d1a5fa4ed2f None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:16.910029 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4a48a2-8b75-45e2-86ef-4d1a5fa4ed2f None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:16.910302 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4a48a2-8b75-45e2-86ef-4d1a5fa4ed2f None admin] PATH_INFO: `/v3/projects` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:16.911045 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4a48a2-8b75-45e2-86ef-4d1a5fa4ed2f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:16.911348 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4a48a2-8b75-45e2-86ef-4d1a5fa4ed2f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:16.912886 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4a48a2-8b75-45e2-86ef-4d1a5fa4ed2f None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:16.946956 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:16.950332 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 273/546] 10.222.0.59 () {56 vars in 1148 bytes} [Tue Oct 11 11:47:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:47:16.953741 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4a48a2-8b75-45e2-86ef-4d1a5fa4ed2f None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:16.981678 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913e1eb8-67fb-4e0d-a270-b83bf2c3c2e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:16.982335 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913e1eb8-67fb-4e0d-a270-b83bf2c3c2e4 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:16.982635 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913e1eb8-67fb-4e0d-a270-b83bf2c3c2e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:16.982916 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913e1eb8-67fb-4e0d-a270-b83bf2c3c2e4 None admin] PATH_INFO: `/v3/domains` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:16.983575 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913e1eb8-67fb-4e0d-a270-b83bf2c3c2e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:16.983880 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913e1eb8-67fb-4e0d-a270-b83bf2c3c2e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:16.985321 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913e1eb8-67fb-4e0d-a270-b83bf2c3c2e4 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:17.019549 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 274/547] 10.222.0.59 () {56 vars in 1109 bytes} [Tue Oct 11 11:47:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 11:47:17.024834 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913e1eb8-67fb-4e0d-a270-b83bf2c3c2e4 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:17.086298 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:17.148705 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4a48a2-8b75-45e2-86ef-4d1a5fa4ed2f None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:17.188391 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913e1eb8-67fb-4e0d-a270-b83bf2c3c2e4 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:17.196931 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dde39f-e006-41f2-a2e7-0cb2b3b3355a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:17.198441 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dde39f-e006-41f2-a2e7-0cb2b3b3355a None admin] REQUEST_METHOD: `PUT` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:17.198866 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dde39f-e006-41f2-a2e7-0cb2b3b3355a None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:17.199241 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dde39f-e006-41f2-a2e7-0cb2b3b3355a None admin] PATH_INFO: `/v3/projects/26ecf552cc9a4e5ca1d380509683dc4a/users/8a19eceb62ce4157a4c5fe1866a89df9/roles/f5a43d8f3fa24a87a7abe7492cf6f87f` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:17.226102 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dde39f-e006-41f2-a2e7-0cb2b3b3355a None admin] RBAC: Authorizing `identity:create_grant(project_id=26ecf552cc9a4e5ca1d380509683dc4a, user_id=8a19eceb62ce4157a4c5fe1866a89df9, role_id=f5a43d8f3fa24a87a7abe7492cf6f87f)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:17.228164 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dde39f-e006-41f2-a2e7-0cb2b3b3355a None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:17.232236 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f658f6fb-d361-4c74-bd28-2ff1ad41d794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:17.232881 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f658f6fb-d361-4c74-bd28-2ff1ad41d794 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:17.233164 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f658f6fb-d361-4c74-bd28-2ff1ad41d794 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:17.233449 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f658f6fb-d361-4c74-bd28-2ff1ad41d794 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:17.244512 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f658f6fb-d361-4c74-bd28-2ff1ad41d794 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:17.247137 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f658f6fb-d361-4c74-bd28-2ff1ad41d794 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:17.247869 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 274/548] 10.222.0.59 () {58 vars in 1439 bytes} [Tue Oct 11 11:47:16 2022] PUT /identity/v3/projects/26ecf552cc9a4e5ca1d380509683dc4a/users/8a19eceb62ce4157a4c5fe1866a89df9/roles/f5a43d8f3fa24a87a7abe7492cf6f87f => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 11:47:17.249010 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f21f6e3-f51b-48d6-a7d2-a1cb4b95df78 None None] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:17.249440 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f21f6e3-f51b-48d6-a7d2-a1cb4b95df78 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:17.249772 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f21f6e3-f51b-48d6-a7d2-a1cb4b95df78 None None] PATH_INFO: `/` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:17.258547 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 275/549] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:47:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:47:17.258547 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a29152f-f8ab-470f-991a-9dd9864520fd None None] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:17.258547 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a29152f-f8ab-470f-991a-9dd9864520fd None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:17.258547 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a29152f-f8ab-470f-991a-9dd9864520fd None None] PATH_INFO: `/` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:17.261063 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 276/550] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:47:17 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:47:17.262650 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fef296-a2ad-4bac-9cda-a6173fc9b304 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:17.263031 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fef296-a2ad-4bac-9cda-a6173fc9b304 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:17.263289 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fef296-a2ad-4bac-9cda-a6173fc9b304 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:17.278287 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application [None req-f658f6fb-d361-4c74-bd28-2ff1ad41d794 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 11 11:47:17.278287 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:47:17.278287 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:47:17.278287 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:47:17.278287 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:47:17.278287 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:47:17.278287 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:47:17.278287 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:47:17.278287 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:47:17.278287 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:47:17.278287 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:47:17.278287 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:47:17.278287 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 11 11:47:17.278287 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 11 11:47:17.278287 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 11 11:47:17.278287 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 11 11:47:17.280306 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:47:17.280306 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:47:17.280306 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 11 11:47:17.280306 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 11:47:17.280306 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 11 11:47:17.280306 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 11:47:17.280306 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:47:17.280306 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:47:17.280306 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:47:17.280306 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:47:17.280306 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:47:17.280306 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application with Lock( Oct 11 11:47:17.280306 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:47:17.280306 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:47:17.280306 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:47:17.280306 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:47:17.280306 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:47:17.280306 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:47:17.282102 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:47:17.282102 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:47:17.282102 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 11 11:47:17.282102 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 11 11:47:17.282102 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 11 11:47:17.282102 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 11 11:47:17.282102 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 11 11:47:17.282102 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 11 11:47:17.282102 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 11 11:47:17.282102 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:47:17.282102 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 275/551] 10.222.0.59 () {56 vars in 1096 bytes} [Tue Oct 11 11:47:17 2022] GET /identity/v3/users/swift => generated 83 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:47:17.286283 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6756a51-6dfb-46c5-b873-eabcea0c2395 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:17.286283 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6756a51-6dfb-46c5-b873-eabcea0c2395 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:17.286283 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6756a51-6dfb-46c5-b873-eabcea0c2395 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:17.295877 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-96fef296-a2ad-4bac-9cda-a6173fc9b304 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:47:17.297418 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-96fef296-a2ad-4bac-9cda-a6173fc9b304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:47:17.321883 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-f6756a51-6dfb-46c5-b873-eabcea0c2395 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:47:17.324661 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-f6756a51-6dfb-46c5-b873-eabcea0c2395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:47:17.385674 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:17.398888 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 277/552] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:47:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 11:47:17.400782 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fef296-a2ad-4bac-9cda-a6173fc9b304 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:17.482596 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 276/553] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:47:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 11 11:47:17.513180 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:17.582169 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fef296-a2ad-4bac-9cda-a6173fc9b304 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:17.622506 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55d8d5c-64ae-4d3e-82c6-528fe12d1377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:17.623177 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55d8d5c-64ae-4d3e-82c6-528fe12d1377 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:17.623459 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55d8d5c-64ae-4d3e-82c6-528fe12d1377 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:17.623739 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55d8d5c-64ae-4d3e-82c6-528fe12d1377 None admin] PATH_INFO: `/v3/users` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:17.624582 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55d8d5c-64ae-4d3e-82c6-528fe12d1377 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:17.624888 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55d8d5c-64ae-4d3e-82c6-528fe12d1377 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:17.626492 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55d8d5c-64ae-4d3e-82c6-528fe12d1377 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:17.648649 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 278/554] 10.222.0.59 () {56 vars in 1135 bytes} [Tue Oct 11 11:47:17 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:47:17.654511 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6756a51-6dfb-46c5-b873-eabcea0c2395 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:17.739411 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55d8d5c-64ae-4d3e-82c6-528fe12d1377 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:17.769998 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6756a51-6dfb-46c5-b873-eabcea0c2395 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:17.812550 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd8e872-2be3-498c-8805-fefcc44ef43d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:17.813193 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd8e872-2be3-498c-8805-fefcc44ef43d None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:17.813472 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd8e872-2be3-498c-8805-fefcc44ef43d None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:17.813724 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd8e872-2be3-498c-8805-fefcc44ef43d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:17.826346 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd8e872-2be3-498c-8805-fefcc44ef43d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:17.827893 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd8e872-2be3-498c-8805-fefcc44ef43d None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:17.846542 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application [None req-4cd8e872-2be3-498c-8805-fefcc44ef43d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 11:47:17.846542 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:47:17.846542 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:47:17.846542 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:47:17.846542 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:47:17.846542 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:47:17.846542 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:47:17.846542 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:47:17.846542 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:47:17.846542 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:47:17.846542 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:47:17.846542 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:47:17.846542 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 11:47:17.846542 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 11:47:17.846542 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 11:47:17.846542 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 11:47:17.848313 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:47:17.848313 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:47:17.848313 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:47:17.848313 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:47:17.848313 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:47:17.848313 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:47:17.848313 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:47:17.848313 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application with Lock( Oct 11 11:47:17.848313 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:47:17.848313 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:47:17.848313 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:47:17.848313 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:47:17.848313 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:47:17.848313 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:47:17.848313 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:47:17.848313 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:47:17.848313 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 11:47:17.850074 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 11:47:17.850074 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 11:47:17.850074 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:47:17.850074 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 277/555] 10.222.0.59 () {56 vars in 1108 bytes} [Tue Oct 11 11:47:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:47:17.853539 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd8e872-2be3-498c-8805-fefcc44ef43d None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:17.872342 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:17.999646 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd8e872-2be3-498c-8805-fefcc44ef43d None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:18.014993 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55d8d5c-64ae-4d3e-82c6-528fe12d1377 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:18.018236 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:18.046827 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d3bf8d-d0a7-4a39-a5fb-77ee6d2ccaef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:18.048119 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d3bf8d-d0a7-4a39-a5fb-77ee6d2ccaef None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:18.048816 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d3bf8d-d0a7-4a39-a5fb-77ee6d2ccaef None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:18.049265 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d3bf8d-d0a7-4a39-a5fb-77ee6d2ccaef None admin] PATH_INFO: `/v3/domains` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:18.050113 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d3bf8d-d0a7-4a39-a5fb-77ee6d2ccaef None admin] RBAC: Adding query filter params (name=Default) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:18.050903 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d3bf8d-d0a7-4a39-a5fb-77ee6d2ccaef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:18.053005 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d3bf8d-d0a7-4a39-a5fb-77ee6d2ccaef None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:18.068895 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 278/556] 10.222.0.59 () {56 vars in 1109 bytes} [Tue Oct 11 11:47:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 11:47:18.070747 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d3bf8d-d0a7-4a39-a5fb-77ee6d2ccaef None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:18.083550 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412f4468-6359-4822-8e3f-0b5a7132c4d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:18.084498 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412f4468-6359-4822-8e3f-0b5a7132c4d6 None admin] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:18.084955 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412f4468-6359-4822-8e3f-0b5a7132c4d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:18.085463 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412f4468-6359-4822-8e3f-0b5a7132c4d6 None admin] PATH_INFO: `/v3/services` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:18.086203 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412f4468-6359-4822-8e3f-0b5a7132c4d6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:18.087813 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412f4468-6359-4822-8e3f-0b5a7132c4d6 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:18.114762 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 279/557] 10.222.0.59 () {60 vars in 1141 bytes} [Tue Oct 11 11:47:17 2022] POST /identity/v3/services => generated 251 bytes in 380 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 11:47:18.116005 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412f4468-6359-4822-8e3f-0b5a7132c4d6 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:18.235111 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:18.282611 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d3bf8d-d0a7-4a39-a5fb-77ee6d2ccaef None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:18.325454 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9edd075-a1c7-4ba8-8044-c46edd5955f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:18.326381 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9edd075-a1c7-4ba8-8044-c46edd5955f1 None admin] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:18.326851 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9edd075-a1c7-4ba8-8044-c46edd5955f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:18.327282 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9edd075-a1c7-4ba8-8044-c46edd5955f1 None admin] PATH_INFO: `/v3/services` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:18.328093 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9edd075-a1c7-4ba8-8044-c46edd5955f1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:18.329741 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9edd075-a1c7-4ba8-8044-c46edd5955f1 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:18.353242 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 279/558] 10.222.0.59 () {60 vars in 1141 bytes} [Tue Oct 11 11:47:18 2022] POST /identity/v3/services => generated 240 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 11:47:18.379249 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412f4468-6359-4822-8e3f-0b5a7132c4d6 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:18.436354 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82f083a-f8cd-44ee-8453-c4d02fa11bde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:18.437290 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82f083a-f8cd-44ee-8453-c4d02fa11bde None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:18.437643 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82f083a-f8cd-44ee-8453-c4d02fa11bde None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:18.438016 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82f083a-f8cd-44ee-8453-c4d02fa11bde None admin] PATH_INFO: `/v3/projects/service` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:18.447408 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82f083a-f8cd-44ee-8453-c4d02fa11bde None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:18.449109 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82f083a-f8cd-44ee-8453-c4d02fa11bde None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:18.463312 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application [None req-f82f083a-f8cd-44ee-8453-c4d02fa11bde None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 11:47:18.463312 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:47:18.463312 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:47:18.463312 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:47:18.463312 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:47:18.463312 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:47:18.463312 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:47:18.463312 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:47:18.463312 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:47:18.463312 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:47:18.463312 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:47:18.463312 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:47:18.463312 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 11 11:47:18.463312 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 11 11:47:18.463312 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 11 11:47:18.464259 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 11 11:47:18.464259 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:47:18.464259 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:47:18.464259 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:47:18.464259 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:47:18.464259 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:47:18.464259 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:47:18.464259 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:47:18.464259 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application with Lock( Oct 11 11:47:18.464259 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:47:18.464259 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:47:18.464259 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:47:18.464259 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:47:18.464259 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:47:18.464259 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:47:18.464259 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:47:18.464259 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:47:18.465265 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 11 11:47:18.465265 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 11 11:47:18.465265 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 11 11:47:18.465265 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 11 11:47:18.465265 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 11 11:47:18.465265 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 11 11:47:18.465265 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 11:47:18.465265 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:47:18.465265 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 280/559] 10.222.0.59 () {56 vars in 1111 bytes} [Tue Oct 11 11:47:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 350 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:47:18.472772 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9edd075-a1c7-4ba8-8044-c46edd5955f1 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:18.680893 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9edd075-a1c7-4ba8-8044-c46edd5955f1 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:18.736754 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:18.746940 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db83efb3-3aae-4b35-949e-466f0215d99d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:18.748487 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db83efb3-3aae-4b35-949e-466f0215d99d None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:18.749180 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db83efb3-3aae-4b35-949e-466f0215d99d None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:18.749966 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db83efb3-3aae-4b35-949e-466f0215d99d None admin] PATH_INFO: `/v3/projects` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:18.751554 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db83efb3-3aae-4b35-949e-466f0215d99d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:18.753766 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db83efb3-3aae-4b35-949e-466f0215d99d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:18.756538 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db83efb3-3aae-4b35-949e-466f0215d99d None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:18.785112 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 280/560] 10.222.0.59 () {56 vars in 1148 bytes} [Tue Oct 11 11:47:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 11:47:18.791561 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82f083a-f8cd-44ee-8453-c4d02fa11bde None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:18.823454 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:18.940917 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82f083a-f8cd-44ee-8453-c4d02fa11bde None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:18.996150 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faedef72-d855-43ff-b2e9-16f83dfd9738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:18.996875 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faedef72-d855-43ff-b2e9-16f83dfd9738 None admin] REQUEST_METHOD: `PUT` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:18.997155 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faedef72-d855-43ff-b2e9-16f83dfd9738 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:18.997430 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faedef72-d855-43ff-b2e9-16f83dfd9738 None admin] PATH_INFO: `/v3/projects/26ecf552cc9a4e5ca1d380509683dc4a/users/76e7fee1993a40d5ba4295e7daeb3e29/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:18.999641 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faedef72-d855-43ff-b2e9-16f83dfd9738 None admin] RBAC: Authorizing `identity:create_grant(project_id=26ecf552cc9a4e5ca1d380509683dc4a, user_id=76e7fee1993a40d5ba4295e7daeb3e29, role_id=5cf1f4bd2fef46ab9c6eff56d52f9f69)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:19.001157 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faedef72-d855-43ff-b2e9-16f83dfd9738 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:19.022160 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 281/561] 10.222.0.59 () {58 vars in 1439 bytes} [Tue Oct 11 11:47:18 2022] PUT /identity/v3/projects/26ecf552cc9a4e5ca1d380509683dc4a/users/76e7fee1993a40d5ba4295e7daeb3e29/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 11:47:19.152225 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9f4b5a-7eb1-4d9b-b3e1-edd488abf216 None None] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:19.152798 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9f4b5a-7eb1-4d9b-b3e1-edd488abf216 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:19.153252 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9f4b5a-7eb1-4d9b-b3e1-edd488abf216 None None] PATH_INFO: `/` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:19.155322 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 281/562] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:47:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:47:19.175021 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beedbcd-2e31-4c45-87c6-a71339cec7e3 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:19.175307 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beedbcd-2e31-4c45-87c6-a71339cec7e3 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:19.175586 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beedbcd-2e31-4c45-87c6-a71339cec7e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:19.200485 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-3beedbcd-2e31-4c45-87c6-a71339cec7e3 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:47:19.201791 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-3beedbcd-2e31-4c45-87c6-a71339cec7e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:47:19.306159 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 282/563] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:47:19 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 11:47:19.599788 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9f4b5a-7eb1-4d9b-b3e1-edd488abf216 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:19.740029 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9f4b5a-7eb1-4d9b-b3e1-edd488abf216 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:19.784995 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d259cc71-1d86-4a96-93b5-15beebb408b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:19.786054 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d259cc71-1d86-4a96-93b5-15beebb408b5 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:19.786998 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d259cc71-1d86-4a96-93b5-15beebb408b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:19.787476 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d259cc71-1d86-4a96-93b5-15beebb408b5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:19.800270 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d259cc71-1d86-4a96-93b5-15beebb408b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 11:47:19.805606 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application [None req-d259cc71-1d86-4a96-93b5-15beebb408b5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 11 11:47:19.805606 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:47:19.805606 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:47:19.805606 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:47:19.805606 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:47:19.805606 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:47:19.805606 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:47:19.805606 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:47:19.805606 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:47:19.805606 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:47:19.805606 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:47:19.805606 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:47:19.805606 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 11:47:19.805606 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 11:47:19.805606 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 11:47:19.805606 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 11:47:19.806616 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 11:47:19.806616 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 11:47:19.806616 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 11:47:19.806616 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 11:47:19.806616 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:47:19.806616 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:47:19.806616 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:47:19.806616 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:47:19.806616 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:47:19.806616 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:47:19.806616 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:47:19.806616 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application with Lock( Oct 11 11:47:19.806616 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:47:19.806616 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:47:19.806616 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:47:19.806616 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:47:19.807459 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:47:19.807459 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:47:19.807459 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:47:19.807459 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:47:19.807459 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 11:47:19.807459 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 11:47:19.807459 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 11:47:19.807459 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 11:47:19.807459 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 11:47:19.807459 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 11:47:19.807459 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 11 11:47:19.807459 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:47:19.810200 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 282/564] 10.222.0.59 () {56 vars in 1096 bytes} [Tue Oct 11 11:47:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:47:19.818600 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3beedbcd-2e31-4c45-87c6-a71339cec7e3 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:19.851053 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:20.088579 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3beedbcd-2e31-4c45-87c6-a71339cec7e3 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:20.155176 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be78e03e-a68e-4dac-8014-bc6f0730fd37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:20.155721 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be78e03e-a68e-4dac-8014-bc6f0730fd37 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:20.155999 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be78e03e-a68e-4dac-8014-bc6f0730fd37 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:20.156278 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be78e03e-a68e-4dac-8014-bc6f0730fd37 None admin] PATH_INFO: `/v3/roles` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:20.156973 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be78e03e-a68e-4dac-8014-bc6f0730fd37 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:20.157280 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be78e03e-a68e-4dac-8014-bc6f0730fd37 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:20.158793 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be78e03e-a68e-4dac-8014-bc6f0730fd37 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:20.168572 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 283/565] 10.222.0.59 () {56 vars in 1099 bytes} [Tue Oct 11 11:47:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:47:20.175305 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d259cc71-1d86-4a96-93b5-15beebb408b5 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:20.320086 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:20.402260 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d259cc71-1d86-4a96-93b5-15beebb408b5 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:20.455524 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3451ece-7fd7-4ab8-bb2e-4ad7ef702ccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:20.456545 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3451ece-7fd7-4ab8-bb2e-4ad7ef702ccd None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:20.457029 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3451ece-7fd7-4ab8-bb2e-4ad7ef702ccd None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:20.457504 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3451ece-7fd7-4ab8-bb2e-4ad7ef702ccd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:20.469825 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3451ece-7fd7-4ab8-bb2e-4ad7ef702ccd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:20.472055 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3451ece-7fd7-4ab8-bb2e-4ad7ef702ccd None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:20.500281 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application [None req-d3451ece-7fd7-4ab8-bb2e-4ad7ef702ccd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 11:47:20.500281 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:47:20.500281 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:47:20.500281 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:47:20.500281 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:47:20.500281 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:47:20.500281 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:47:20.500281 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:47:20.500281 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:47:20.500281 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:47:20.500281 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:47:20.500281 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:47:20.500281 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 11:47:20.500281 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 11:47:20.500281 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 11:47:20.500281 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 11:47:20.501302 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:47:20.501302 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:47:20.501302 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:47:20.501302 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:47:20.501302 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:47:20.501302 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:47:20.501302 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:47:20.501302 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application with Lock( Oct 11 11:47:20.501302 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:47:20.501302 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:47:20.501302 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:47:20.501302 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:47:20.501302 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:47:20.501302 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:47:20.501302 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:47:20.501302 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:47:20.501302 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 11:47:20.502226 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 11:47:20.502226 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 11:47:20.502226 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:47:20.504139 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 283/566] 10.222.0.59 () {56 vars in 1108 bytes} [Tue Oct 11 11:47:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:47:20.515798 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be78e03e-a68e-4dac-8014-bc6f0730fd37 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:20.550141 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ef92aa-2a24-4729-ae2b-1fce32fbda05 None None] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:20.550818 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ef92aa-2a24-4729-ae2b-1fce32fbda05 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:20.551330 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ef92aa-2a24-4729-ae2b-1fce32fbda05 None None] PATH_INFO: `/` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:20.553149 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 284/567] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:47:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 11 11:47:20.563854 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b713015-a5de-4a82-8c01-044a446665e6 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:20.564687 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b713015-a5de-4a82-8c01-044a446665e6 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:20.565166 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b713015-a5de-4a82-8c01-044a446665e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:20.622958 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:20.628328 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-0b713015-a5de-4a82-8c01-044a446665e6 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:47:20.630380 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-0b713015-a5de-4a82-8c01-044a446665e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:47:20.642457 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be78e03e-a68e-4dac-8014-bc6f0730fd37 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:20.711958 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50601b1c-98eb-4676-b754-ea7a0f58356b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:20.713042 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50601b1c-98eb-4676-b754-ea7a0f58356b None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:20.713329 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50601b1c-98eb-4676-b754-ea7a0f58356b None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:20.713602 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50601b1c-98eb-4676-b754-ea7a0f58356b None admin] PATH_INFO: `/v3/domains` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:20.714267 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50601b1c-98eb-4676-b754-ea7a0f58356b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:20.714586 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50601b1c-98eb-4676-b754-ea7a0f58356b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:20.716002 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50601b1c-98eb-4676-b754-ea7a0f58356b None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:20.745222 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 284/568] 10.222.0.59 () {56 vars in 1109 bytes} [Tue Oct 11 11:47:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:47:20.754187 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50601b1c-98eb-4676-b754-ea7a0f58356b None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:20.797005 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 285/569] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:47:20 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 11 11:47:20.883811 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50601b1c-98eb-4676-b754-ea7a0f58356b None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:20.916185 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:20.922490 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a754be-d200-472b-9a9e-420ba007f6b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:20.927492 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a754be-d200-472b-9a9e-420ba007f6b0 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:20.927776 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a754be-d200-472b-9a9e-420ba007f6b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:20.928053 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a754be-d200-472b-9a9e-420ba007f6b0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:20.938754 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a754be-d200-472b-9a9e-420ba007f6b0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:20.940454 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a754be-d200-472b-9a9e-420ba007f6b0 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:20.960068 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application [None req-35a754be-d200-472b-9a9e-420ba007f6b0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 11 11:47:20.960068 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:47:20.960068 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:47:20.960068 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:47:20.960068 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:47:20.960068 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:47:20.960068 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:47:20.960068 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:47:20.960068 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:47:20.960068 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:47:20.960068 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:47:20.960068 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:47:20.960068 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 11 11:47:20.960068 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 11 11:47:20.960068 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 11 11:47:20.960068 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 11 11:47:20.961088 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:47:20.961088 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:47:20.961088 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 11 11:47:20.961088 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 11:47:20.961088 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 11 11:47:20.961088 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 11:47:20.961088 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:47:20.961088 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:47:20.961088 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:47:20.961088 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:47:20.961088 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:47:20.961088 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application with Lock( Oct 11 11:47:20.961088 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:47:20.961088 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:47:20.961088 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:47:20.961088 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:47:20.961088 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:47:20.961088 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:47:20.962157 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:47:20.962157 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:47:20.962157 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 11 11:47:20.962157 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 11 11:47:20.962157 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 11 11:47:20.962157 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 11 11:47:20.962157 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 11 11:47:20.962157 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 11 11:47:20.962157 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 11 11:47:20.962157 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:47:20.966871 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 285/570] 10.222.0.59 () {56 vars in 1093 bytes} [Tue Oct 11 11:47:20 2022] GET /identity/v3/users/nova => generated 82 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 11 11:47:20.972933 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b713015-a5de-4a82-8c01-044a446665e6 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:21.055708 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:21.142370 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b713015-a5de-4a82-8c01-044a446665e6 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:21.197456 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be8e804-1a6f-41f4-a79a-1bed913baca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:21.198112 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be8e804-1a6f-41f4-a79a-1bed913baca8 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:21.198395 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be8e804-1a6f-41f4-a79a-1bed913baca8 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:21.198692 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be8e804-1a6f-41f4-a79a-1bed913baca8 None admin] PATH_INFO: `/v3/users` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:21.199466 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be8e804-1a6f-41f4-a79a-1bed913baca8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:21.199774 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be8e804-1a6f-41f4-a79a-1bed913baca8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:21.201289 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be8e804-1a6f-41f4-a79a-1bed913baca8 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:21.229029 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 286/571] 10.222.0.59 () {56 vars in 1133 bytes} [Tue Oct 11 11:47:20 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 11:47:21.237510 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a754be-d200-472b-9a9e-420ba007f6b0 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:21.304384 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160debb7-e037-4f22-8742-509c2697458a None None] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:21.304848 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160debb7-e037-4f22-8742-509c2697458a None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:21.305260 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160debb7-e037-4f22-8742-509c2697458a None None] PATH_INFO: `/` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:21.307004 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 287/572] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:47:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:47:21.313634 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f4e95f-c00e-40ab-87d4-a2a44924005c None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:21.314150 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f4e95f-c00e-40ab-87d4-a2a44924005c None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:21.314721 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f4e95f-c00e-40ab-87d4-a2a44924005c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:21.372231 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-47f4e95f-c00e-40ab-87d4-a2a44924005c None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:47:21.374147 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-47f4e95f-c00e-40ab-87d4-a2a44924005c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:47:21.400206 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a754be-d200-472b-9a9e-420ba007f6b0 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:21.443716 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02607bff-36d5-416c-9cf3-62ff01fa3f8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:21.444360 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02607bff-36d5-416c-9cf3-62ff01fa3f8d None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:21.444641 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02607bff-36d5-416c-9cf3-62ff01fa3f8d None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:21.444910 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02607bff-36d5-416c-9cf3-62ff01fa3f8d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:21.516420 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:21.528364 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02607bff-36d5-416c-9cf3-62ff01fa3f8d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:21.530742 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02607bff-36d5-416c-9cf3-62ff01fa3f8d None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:21.593588 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 288/573] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:47:21 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 11:47:21.595687 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f4e95f-c00e-40ab-87d4-a2a44924005c None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:21.638045 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:21.649054 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 286/574] 10.222.0.59 () {58 vars in 1282 bytes} [Tue Oct 11 11:47:21 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 11 11:47:21.656348 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02607bff-36d5-416c-9cf3-62ff01fa3f8d None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:21.891970 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f4e95f-c00e-40ab-87d4-a2a44924005c None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:21.903230 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:21.920567 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02607bff-36d5-416c-9cf3-62ff01fa3f8d None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:21.963503 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1b33a6-818a-4eaf-b70c-394c9910b774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:21.964177 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1b33a6-818a-4eaf-b70c-394c9910b774 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:21.964455 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1b33a6-818a-4eaf-b70c-394c9910b774 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:21.964719 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1b33a6-818a-4eaf-b70c-394c9910b774 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:21.971812 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa120fab-23be-4f88-88d0-a45b405d441c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:21.972526 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa120fab-23be-4f88-88d0-a45b405d441c None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:21.972810 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa120fab-23be-4f88-88d0-a45b405d441c None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:21.973083 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa120fab-23be-4f88-88d0-a45b405d441c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:21.978836 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1b33a6-818a-4eaf-b70c-394c9910b774 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 11:47:21.984733 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa120fab-23be-4f88-88d0-a45b405d441c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:21.985200 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application [None req-7c1b33a6-818a-4eaf-b70c-394c9910b774 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 11 11:47:21.985200 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:47:21.985200 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:47:21.985200 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:47:21.985200 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:47:21.985200 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:47:21.985200 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:47:21.985200 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:47:21.985200 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:47:21.985200 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:47:21.985200 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:47:21.985200 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:47:21.985200 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 11:47:21.985200 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 11:47:21.985200 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 11:47:21.985200 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 11:47:21.987250 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 11:47:21.987250 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 11:47:21.987250 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 11:47:21.987250 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 11:47:21.987250 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:47:21.987250 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:47:21.987250 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:47:21.987250 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:47:21.987250 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:47:21.987250 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:47:21.987250 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:47:21.987250 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application with Lock( Oct 11 11:47:21.987250 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:47:21.987250 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:47:21.987250 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:47:21.987250 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:47:21.988904 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:47:21.988904 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:47:21.988904 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:47:21.988904 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:47:21.988904 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 11:47:21.988904 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 11:47:21.988904 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 11:47:21.988904 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 11:47:21.988904 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 11:47:21.988904 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 11:47:21.988904 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 11 11:47:21.988904 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:47:21.990169 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa120fab-23be-4f88-88d0-a45b405d441c None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:21.990314 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 289/575] 10.222.0.59 () {56 vars in 1102 bytes} [Tue Oct 11 11:47:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 392 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:47:21.990314 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1b33a6-818a-4eaf-b70c-394c9910b774 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:22.010135 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application [None req-aa120fab-23be-4f88-88d0-a45b405d441c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 11:47:22.010135 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:47:22.010135 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:47:22.010135 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:47:22.010135 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:47:22.010135 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:47:22.010135 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:47:22.010135 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:47:22.010135 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:47:22.010135 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:47:22.010135 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:47:22.010135 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:47:22.010135 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 11:47:22.010135 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 11:47:22.010135 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 11:47:22.010135 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 11:47:22.011991 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:47:22.011991 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:47:22.011991 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:47:22.011991 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:47:22.011991 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:47:22.011991 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:47:22.011991 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:47:22.011991 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application with Lock( Oct 11 11:47:22.011991 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:47:22.011991 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:47:22.011991 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:47:22.011991 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:47:22.011991 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:47:22.011991 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:47:22.011991 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:47:22.011991 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:47:22.011991 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 11:47:22.013684 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 11:47:22.013684 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 11:47:22.013684 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:47:22.017573 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 287/576] 10.222.0.59 () {56 vars in 1108 bytes} [Tue Oct 11 11:47:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 362 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:47:22.019653 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa120fab-23be-4f88-88d0-a45b405d441c None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:22.097241 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:22.120670 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1b33a6-818a-4eaf-b70c-394c9910b774 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:22.164311 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c432cabe-4858-47f3-919d-45618095af95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:22.165137 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c432cabe-4858-47f3-919d-45618095af95 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:22.165419 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c432cabe-4858-47f3-919d-45618095af95 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:22.165684 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c432cabe-4858-47f3-919d-45618095af95 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:22.166780 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c432cabe-4858-47f3-919d-45618095af95 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:22.168217 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c432cabe-4858-47f3-919d-45618095af95 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:22.182202 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application [None req-c432cabe-4858-47f3-919d-45618095af95 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 11 11:47:22.182202 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:47:22.182202 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 11 11:47:22.182202 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 11 11:47:22.182202 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 11 11:47:22.182202 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 11 11:47:22.182202 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 11 11:47:22.182202 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 11:47:22.182202 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 11 11:47:22.182202 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:47:22.182202 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 11 11:47:22.182202 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:47:22.182202 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:47:22.182202 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:47:22.182202 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:47:22.182202 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:47:22.182202 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:47:22.184322 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:47:22.184322 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:47:22.184322 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:47:22.184322 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:47:22.184322 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:47:22.184322 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:47:22.184322 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 11 11:47:22.184322 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 11 11:47:22.184322 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 11 11:47:22.184322 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 11 11:47:22.184322 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:47:22.184322 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:47:22.184322 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:47:22.184322 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:47:22.184322 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:47:22.184322 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:47:22.186070 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:47:22.186070 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application with Lock( Oct 11 11:47:22.186070 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:47:22.186070 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:47:22.186070 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:47:22.186070 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:47:22.186070 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:47:22.186070 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:47:22.186070 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:47:22.186070 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:47:22.186070 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 11 11:47:22.186070 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 11:47:22.186070 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 11 11:47:22.186070 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:47:22.187586 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 290/577] 10.222.0.59 () {56 vars in 1129 bytes} [Tue Oct 11 11:47:21 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:47:22.188590 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c432cabe-4858-47f3-919d-45618095af95 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:22.199721 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa120fab-23be-4f88-88d0-a45b405d441c None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:22.250004 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f10cbf-b3a5-41a9-91dd-7fa892bdcd6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:22.251082 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f10cbf-b3a5-41a9-91dd-7fa892bdcd6d None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:22.251779 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f10cbf-b3a5-41a9-91dd-7fa892bdcd6d None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:22.252279 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f10cbf-b3a5-41a9-91dd-7fa892bdcd6d None admin] PATH_INFO: `/v3/roles` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:22.253515 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f10cbf-b3a5-41a9-91dd-7fa892bdcd6d None admin] RBAC: Adding query filter params (name=service) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:22.254064 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f10cbf-b3a5-41a9-91dd-7fa892bdcd6d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:22.256769 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f10cbf-b3a5-41a9-91dd-7fa892bdcd6d None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:22.272414 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 288/578] 10.222.0.59 () {56 vars in 1103 bytes} [Tue Oct 11 11:47:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:47:22.274621 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a8ef98-09ec-48fb-98a5-9331bf08af2b None None] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:22.275077 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a8ef98-09ec-48fb-98a5-9331bf08af2b None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:22.275534 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a8ef98-09ec-48fb-98a5-9331bf08af2b None None] PATH_INFO: `/` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:22.278044 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 289/579] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:47:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:47:22.279480 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a8ef98-09ec-48fb-98a5-9331bf08af2b None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:22.286210 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:22.336100 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c432cabe-4858-47f3-919d-45618095af95 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:22.376121 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec0847e-1d6b-4355-a3a6-69819d4dcea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:22.376799 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec0847e-1d6b-4355-a3a6-69819d4dcea9 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:22.377076 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec0847e-1d6b-4355-a3a6-69819d4dcea9 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:22.377344 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec0847e-1d6b-4355-a3a6-69819d4dcea9 None admin] PATH_INFO: `/v3/domains` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:22.377985 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec0847e-1d6b-4355-a3a6-69819d4dcea9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:22.378286 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec0847e-1d6b-4355-a3a6-69819d4dcea9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:22.379737 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec0847e-1d6b-4355-a3a6-69819d4dcea9 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:22.392313 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 291/580] 10.222.0.59 () {56 vars in 1109 bytes} [Tue Oct 11 11:47:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:47:22.393213 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec0847e-1d6b-4355-a3a6-69819d4dcea9 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:22.401419 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:22.419941 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a8ef98-09ec-48fb-98a5-9331bf08af2b None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:22.466842 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933cbaed-61dd-48dc-97e8-40363a2eaab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:22.467498 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933cbaed-61dd-48dc-97e8-40363a2eaab6 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:22.467778 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933cbaed-61dd-48dc-97e8-40363a2eaab6 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:22.468031 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933cbaed-61dd-48dc-97e8-40363a2eaab6 None admin] PATH_INFO: `/v3/services` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:22.468745 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933cbaed-61dd-48dc-97e8-40363a2eaab6 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:22.469028 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933cbaed-61dd-48dc-97e8-40363a2eaab6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:22.470722 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933cbaed-61dd-48dc-97e8-40363a2eaab6 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:22.477542 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 290/581] 10.222.0.59 () {56 vars in 1124 bytes} [Tue Oct 11 11:47:22 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:47:22.480151 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cae9aee-eb7e-4ce7-830c-7dcf5a0a728b None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:22.480535 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cae9aee-eb7e-4ce7-830c-7dcf5a0a728b None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:22.480718 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cae9aee-eb7e-4ce7-830c-7dcf5a0a728b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:22.500675 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-6cae9aee-eb7e-4ce7-830c-7dcf5a0a728b None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:47:22.502080 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-6cae9aee-eb7e-4ce7-830c-7dcf5a0a728b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:47:22.518595 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec0847e-1d6b-4355-a3a6-69819d4dcea9 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:22.558480 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd16689-e4ef-4fff-8e70-7adb47c1e980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:22.559183 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd16689-e4ef-4fff-8e70-7adb47c1e980 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:22.559474 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd16689-e4ef-4fff-8e70-7adb47c1e980 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:22.559724 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd16689-e4ef-4fff-8e70-7adb47c1e980 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:22.572537 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd16689-e4ef-4fff-8e70-7adb47c1e980 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:22.574028 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd16689-e4ef-4fff-8e70-7adb47c1e980 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:22.585608 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:22.594097 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application [None req-5cd16689-e4ef-4fff-8e70-7adb47c1e980 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 11:47:22.594097 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:47:22.594097 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:47:22.594097 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:47:22.594097 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:47:22.594097 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:47:22.594097 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:47:22.594097 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:47:22.594097 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:47:22.594097 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:47:22.594097 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:47:22.594097 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:47:22.594097 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 11:47:22.594097 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 11:47:22.594097 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 11:47:22.594097 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 11:47:22.595361 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:47:22.595361 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:47:22.595361 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:47:22.595361 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:47:22.595361 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:47:22.595361 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:47:22.595361 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:47:22.595361 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application with Lock( Oct 11 11:47:22.595361 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:47:22.595361 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:47:22.595361 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:47:22.595361 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:47:22.595361 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:47:22.595361 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:47:22.595361 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:47:22.595361 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:47:22.595361 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 11:47:22.596324 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 11:47:22.596324 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 11:47:22.596324 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:47:22.596499 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 291/582] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:47:22 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 11:47:22.596580 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 292/583] 10.222.0.59 () {56 vars in 1108 bytes} [Tue Oct 11 11:47:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:47:22.597698 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cae9aee-eb7e-4ce7-830c-7dcf5a0a728b None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:22.598081 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba8a17b-4e13-49dc-a0ff-0791ae2af9b3 None None] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:22.598704 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba8a17b-4e13-49dc-a0ff-0791ae2af9b3 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:22.599144 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba8a17b-4e13-49dc-a0ff-0791ae2af9b3 None None] PATH_INFO: `/` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:22.602721 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 293/584] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:47:22 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:47:22.605286 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba8a17b-4e13-49dc-a0ff-0791ae2af9b3 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:22.703712 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:22.777195 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cae9aee-eb7e-4ce7-830c-7dcf5a0a728b None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:22.777468 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba8a17b-4e13-49dc-a0ff-0791ae2af9b3 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:22.821483 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557479e5-8c66-45d4-9de0-961df860f964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:22.822133 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557479e5-8c66-45d4-9de0-961df860f964 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:22.822410 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557479e5-8c66-45d4-9de0-961df860f964 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:22.822724 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557479e5-8c66-45d4-9de0-961df860f964 None admin] PATH_INFO: `/v3/services` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:22.823366 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557479e5-8c66-45d4-9de0-961df860f964 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:22.823667 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557479e5-8c66-45d4-9de0-961df860f964 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:22.823830 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ab07f0-bd0c-4a7a-af0e-7ab85026895d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:22.824518 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ab07f0-bd0c-4a7a-af0e-7ab85026895d None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:22.824976 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ab07f0-bd0c-4a7a-af0e-7ab85026895d None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:22.825228 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557479e5-8c66-45d4-9de0-961df860f964 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:22.825469 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ab07f0-bd0c-4a7a-af0e-7ab85026895d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:22.832077 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 294/585] 10.222.0.59 () {56 vars in 1124 bytes} [Tue Oct 11 11:47:22 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:47:22.835057 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ab07f0-bd0c-4a7a-af0e-7ab85026895d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:22.836988 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ab07f0-bd0c-4a7a-af0e-7ab85026895d None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:22.837586 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557479e5-8c66-45d4-9de0-961df860f964 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:22.852427 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application [None req-31ab07f0-bd0c-4a7a-af0e-7ab85026895d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 11:47:22.852427 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:47:22.852427 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:47:22.852427 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:47:22.852427 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:47:22.852427 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:47:22.852427 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:47:22.852427 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:47:22.852427 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:47:22.852427 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:47:22.852427 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:47:22.852427 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:47:22.852427 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 11 11:47:22.852427 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 11 11:47:22.852427 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 11 11:47:22.854226 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 11 11:47:22.854226 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:47:22.854226 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:47:22.854226 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:47:22.854226 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:47:22.854226 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:47:22.854226 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:47:22.854226 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:47:22.854226 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application with Lock( Oct 11 11:47:22.854226 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:47:22.854226 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:47:22.854226 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:47:22.854226 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:47:22.854226 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:47:22.854226 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:47:22.854226 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:47:22.854226 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:47:22.856975 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 11 11:47:22.856975 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 11 11:47:22.856975 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 11 11:47:22.856975 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 11 11:47:22.856975 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 11 11:47:22.856975 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 11 11:47:22.856975 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 11:47:22.856975 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:47:22.856975 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 292/586] 10.222.0.59 () {56 vars in 1111 bytes} [Tue Oct 11 11:47:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:47:22.856975 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4274242-4a18-40aa-b98a-bff2669eaea5 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:22.858060 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4274242-4a18-40aa-b98a-bff2669eaea5 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:22.858060 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4274242-4a18-40aa-b98a-bff2669eaea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:22.879487 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-f4274242-4a18-40aa-b98a-bff2669eaea5 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:47:22.880730 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-f4274242-4a18-40aa-b98a-bff2669eaea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:47:22.946238 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:22.960414 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557479e5-8c66-45d4-9de0-961df860f964 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:22.968111 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 293/587] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:47:22 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 11:47:22.968111 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4274242-4a18-40aa-b98a-bff2669eaea5 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:22.984566 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:23.003644 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacfb1fa-1229-4152-a0a2-60f5a7d959fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:23.004276 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacfb1fa-1229-4152-a0a2-60f5a7d959fa None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:23.004577 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacfb1fa-1229-4152-a0a2-60f5a7d959fa None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:23.004828 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacfb1fa-1229-4152-a0a2-60f5a7d959fa None admin] PATH_INFO: `/v3/domains` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:23.005462 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacfb1fa-1229-4152-a0a2-60f5a7d959fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:23.005769 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacfb1fa-1229-4152-a0a2-60f5a7d959fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:23.007553 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacfb1fa-1229-4152-a0a2-60f5a7d959fa None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:23.025226 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 295/588] 10.222.0.59 () {56 vars in 1109 bytes} [Tue Oct 11 11:47:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:47:23.026777 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacfb1fa-1229-4152-a0a2-60f5a7d959fa None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:23.121653 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4274242-4a18-40aa-b98a-bff2669eaea5 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:23.179314 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e707d9-b69c-4ac8-982d-993108e23229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:23.180627 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e707d9-b69c-4ac8-982d-993108e23229 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:23.180990 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e707d9-b69c-4ac8-982d-993108e23229 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:23.181442 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e707d9-b69c-4ac8-982d-993108e23229 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:23.182684 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e707d9-b69c-4ac8-982d-993108e23229 None admin] RBAC: Adding query filter params (interface=public, service_id=d4d2f8375b4e49bcb476a5adfddd8cc8, region_id=RegionOne) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:23.183204 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e707d9-b69c-4ac8-982d-993108e23229 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:23.185262 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e707d9-b69c-4ac8-982d-993108e23229 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:23.195538 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacfb1fa-1229-4152-a0a2-60f5a7d959fa None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:23.205431 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 294/589] 10.222.0.59 () {56 vars in 1251 bytes} [Tue Oct 11 11:47:22 2022] GET /identity/v3/endpoints?service_id=d4d2f8375b4e49bcb476a5adfddd8cc8&interface=public®ion_id=RegionOne => generated 194 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:47:23.205431 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e707d9-b69c-4ac8-982d-993108e23229 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:23.257630 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db8ea96-b445-4bf0-88f2-d2b4b5d6c5da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:23.258297 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db8ea96-b445-4bf0-88f2-d2b4b5d6c5da None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:23.258725 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db8ea96-b445-4bf0-88f2-d2b4b5d6c5da None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:23.259089 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db8ea96-b445-4bf0-88f2-d2b4b5d6c5da None admin] PATH_INFO: `/v3/projects` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:23.259885 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db8ea96-b445-4bf0-88f2-d2b4b5d6c5da None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:23.260280 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db8ea96-b445-4bf0-88f2-d2b4b5d6c5da None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:23.261884 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db8ea96-b445-4bf0-88f2-d2b4b5d6c5da None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:23.276693 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 296/590] 10.222.0.59 () {56 vars in 1148 bytes} [Tue Oct 11 11:47:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:47:23.278088 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db8ea96-b445-4bf0-88f2-d2b4b5d6c5da None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:23.280866 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:23.338257 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:23.343992 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e707d9-b69c-4ac8-982d-993108e23229 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:23.386534 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a0c158-7a24-4f49-9cc5-67a9c2a7fe5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:23.387332 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a0c158-7a24-4f49-9cc5-67a9c2a7fe5b None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:23.387794 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a0c158-7a24-4f49-9cc5-67a9c2a7fe5b None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:23.388147 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a0c158-7a24-4f49-9cc5-67a9c2a7fe5b None admin] PATH_INFO: `/v3/services/network` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:23.388906 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a0c158-7a24-4f49-9cc5-67a9c2a7fe5b None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:23.390529 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a0c158-7a24-4f49-9cc5-67a9c2a7fe5b None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:23.401711 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application [None req-09a0c158-7a24-4f49-9cc5-67a9c2a7fe5b None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 11 11:47:23.401711 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:47:23.401711 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 11 11:47:23.401711 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 11 11:47:23.401711 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 11 11:47:23.401711 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 11 11:47:23.401711 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 11 11:47:23.401711 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 11:47:23.401711 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 11 11:47:23.401711 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:47:23.401711 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 11 11:47:23.401711 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:47:23.401711 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:47:23.401711 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:47:23.401711 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:47:23.401711 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:47:23.401711 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:47:23.403939 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:47:23.403939 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:47:23.403939 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:47:23.403939 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:47:23.403939 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:47:23.403939 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:47:23.403939 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 11 11:47:23.403939 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 11 11:47:23.403939 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 11 11:47:23.403939 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 11 11:47:23.403939 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:47:23.403939 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:47:23.403939 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:47:23.403939 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:47:23.403939 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:47:23.403939 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:47:23.405709 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:47:23.405709 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application with Lock( Oct 11 11:47:23.405709 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:47:23.405709 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:47:23.405709 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:47:23.405709 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:47:23.405709 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:47:23.405709 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:47:23.405709 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:47:23.405709 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:47:23.405709 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 11 11:47:23.405709 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 11:47:23.405709 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 11 11:47:23.405709 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:47:23.405709 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 295/591] 10.222.0.59 () {56 vars in 1111 bytes} [Tue Oct 11 11:47:23 2022] GET /identity/v3/services/network => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:47:23.407267 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a0c158-7a24-4f49-9cc5-67a9c2a7fe5b None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:23.416327 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db8ea96-b445-4bf0-88f2-d2b4b5d6c5da None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:23.456285 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87b640c-7242-4e0c-bf78-48305c628568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:23.457050 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87b640c-7242-4e0c-bf78-48305c628568 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:23.457415 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87b640c-7242-4e0c-bf78-48305c628568 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:23.458260 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87b640c-7242-4e0c-bf78-48305c628568 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:23.467784 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87b640c-7242-4e0c-bf78-48305c628568 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:23.469826 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87b640c-7242-4e0c-bf78-48305c628568 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:23.483986 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application [None req-d87b640c-7242-4e0c-bf78-48305c628568 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 11 11:47:23.483986 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:47:23.483986 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:47:23.483986 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:47:23.483986 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:47:23.483986 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:47:23.483986 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:47:23.483986 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:47:23.483986 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:47:23.483986 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:47:23.483986 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:47:23.483986 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:47:23.483986 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 11 11:47:23.483986 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 11 11:47:23.483986 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 11 11:47:23.483986 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 11 11:47:23.486179 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:47:23.486179 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:47:23.486179 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 11 11:47:23.486179 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 11:47:23.486179 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 11 11:47:23.486179 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 11:47:23.486179 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:47:23.486179 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:47:23.486179 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:47:23.486179 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:47:23.486179 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:47:23.486179 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application with Lock( Oct 11 11:47:23.486179 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:47:23.486179 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:47:23.486179 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:47:23.486179 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:47:23.486179 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:47:23.486179 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:47:23.488427 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:47:23.488427 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:47:23.488427 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 11 11:47:23.488427 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 11 11:47:23.488427 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 11 11:47:23.488427 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 11 11:47:23.488427 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 11 11:47:23.488427 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 11 11:47:23.488427 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 11 11:47:23.488427 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:47:23.488427 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 297/592] 10.222.0.59 () {56 vars in 1117 bytes} [Tue Oct 11 11:47:23 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:47:23.488427 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87b640c-7242-4e0c-bf78-48305c628568 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:23.532805 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a0c158-7a24-4f49-9cc5-67a9c2a7fe5b None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:23.553256 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:23.574190 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134c9b7f-f362-4720-814b-90c52ce96a30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:23.575070 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134c9b7f-f362-4720-814b-90c52ce96a30 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:23.575453 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134c9b7f-f362-4720-814b-90c52ce96a30 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:23.575890 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134c9b7f-f362-4720-814b-90c52ce96a30 None admin] PATH_INFO: `/v3/services` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:23.576898 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134c9b7f-f362-4720-814b-90c52ce96a30 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:23.578385 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134c9b7f-f362-4720-814b-90c52ce96a30 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:23.584806 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 296/593] 10.222.0.59 () {56 vars in 1087 bytes} [Tue Oct 11 11:47:23 2022] GET /identity/v3/services => generated 772 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:47:23.586043 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134c9b7f-f362-4720-814b-90c52ce96a30 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:23.611725 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:23.616653 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87b640c-7242-4e0c-bf78-48305c628568 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:23.663822 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890d62d7-13c5-462a-8a24-568c14fd8cfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:23.664668 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890d62d7-13c5-462a-8a24-568c14fd8cfb None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:23.665060 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890d62d7-13c5-462a-8a24-568c14fd8cfb None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:23.665444 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890d62d7-13c5-462a-8a24-568c14fd8cfb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:23.679376 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890d62d7-13c5-462a-8a24-568c14fd8cfb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 11:47:23.685720 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application [None req-890d62d7-13c5-462a-8a24-568c14fd8cfb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 11 11:47:23.685720 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:47:23.685720 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:47:23.685720 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:47:23.685720 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:47:23.685720 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:47:23.685720 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:47:23.685720 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:47:23.685720 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:47:23.685720 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:47:23.685720 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:47:23.685720 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:47:23.685720 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 11:47:23.685720 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 11:47:23.685720 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 11:47:23.685720 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 11:47:23.687735 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 11:47:23.687735 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 11:47:23.687735 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 11:47:23.687735 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 11:47:23.687735 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:47:23.687735 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:47:23.687735 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:47:23.687735 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:47:23.687735 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:47:23.687735 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:47:23.687735 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:47:23.687735 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application with Lock( Oct 11 11:47:23.687735 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:47:23.687735 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:47:23.687735 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:47:23.687735 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:47:23.689502 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:47:23.689502 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:47:23.689502 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:47:23.689502 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:47:23.689502 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 11:47:23.689502 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 11:47:23.689502 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 11:47:23.689502 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 11:47:23.689502 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 11:47:23.689502 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 11:47:23.689502 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 11 11:47:23.689502 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:47:23.689502 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 298/594] 10.222.0.59 () {56 vars in 1096 bytes} [Tue Oct 11 11:47:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:47:23.691094 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890d62d7-13c5-462a-8a24-568c14fd8cfb None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:23.757940 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134c9b7f-f362-4720-814b-90c52ce96a30 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:23.820097 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890d62d7-13c5-462a-8a24-568c14fd8cfb None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:23.828810 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffcdc64-3912-44c9-8202-1c9cf2e1e10b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:23.830048 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffcdc64-3912-44c9-8202-1c9cf2e1e10b None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:23.830663 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffcdc64-3912-44c9-8202-1c9cf2e1e10b None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:23.831222 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffcdc64-3912-44c9-8202-1c9cf2e1e10b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:23.832633 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffcdc64-3912-44c9-8202-1c9cf2e1e10b None admin] RBAC: Adding query filter params (role.id=5cf1f4bd2fef46ab9c6eff56d52f9f69, scope.project.id=26ecf552cc9a4e5ca1d380509683dc4a, user.id=bad5e1cb306b46818096940acc3810b2) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:23.833251 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffcdc64-3912-44c9-8202-1c9cf2e1e10b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:23.836089 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffcdc64-3912-44c9-8202-1c9cf2e1e10b None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:23.847987 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 297/595] 10.222.0.59 () {56 vars in 1374 bytes} [Tue Oct 11 11:47:23 2022] GET /identity/v3/role_assignments?user.id=bad5e1cb306b46818096940acc3810b2&scope.project.id=26ecf552cc9a4e5ca1d380509683dc4a&role.id=5cf1f4bd2fef46ab9c6eff56d52f9f69 => generated 608 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:47:23.849573 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffcdc64-3912-44c9-8202-1c9cf2e1e10b None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:23.863037 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a35d2f-4c53-4e6e-b722-62e4cfb1973a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:23.863767 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a35d2f-4c53-4e6e-b722-62e4cfb1973a None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:23.864053 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a35d2f-4c53-4e6e-b722-62e4cfb1973a None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:23.864330 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a35d2f-4c53-4e6e-b722-62e4cfb1973a None admin] PATH_INFO: `/v3/services` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:23.864996 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a35d2f-4c53-4e6e-b722-62e4cfb1973a None admin] RBAC: Adding query filter params (name=network) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:23.865311 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a35d2f-4c53-4e6e-b722-62e4cfb1973a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:23.868152 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a35d2f-4c53-4e6e-b722-62e4cfb1973a None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:23.875655 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 299/596] 10.222.0.59 () {56 vars in 1112 bytes} [Tue Oct 11 11:47:23 2022] GET /identity/v3/services?name=network => generated 124 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:47:23.876647 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a35d2f-4c53-4e6e-b722-62e4cfb1973a None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:23.937829 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:23.940320 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:24.073142 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffcdc64-3912-44c9-8202-1c9cf2e1e10b None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:24.096014 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a35d2f-4c53-4e6e-b722-62e4cfb1973a None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:24.154269 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab9e4aa-236a-48e3-9670-f093e8505a3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:24.155485 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab9e4aa-236a-48e3-9670-f093e8505a3d None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:24.156067 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab9e4aa-236a-48e3-9670-f093e8505a3d None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:24.156743 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab9e4aa-236a-48e3-9670-f093e8505a3d None admin] PATH_INFO: `/v3/users` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:24.158259 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab9e4aa-236a-48e3-9670-f093e8505a3d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:24.158729 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7215a6e4-2cec-47f7-ae6b-0b82985e4efa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:24.159075 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab9e4aa-236a-48e3-9670-f093e8505a3d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:24.160049 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7215a6e4-2cec-47f7-ae6b-0b82985e4efa None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:24.160508 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7215a6e4-2cec-47f7-ae6b-0b82985e4efa None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:24.160959 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7215a6e4-2cec-47f7-ae6b-0b82985e4efa None admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:24.162043 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7215a6e4-2cec-47f7-ae6b-0b82985e4efa None admin] RBAC: Adding query filter params (name=admin) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:24.162431 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7215a6e4-2cec-47f7-ae6b-0b82985e4efa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:24.162870 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab9e4aa-236a-48e3-9670-f093e8505a3d None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:24.164122 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7215a6e4-2cec-47f7-ae6b-0b82985e4efa None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:24.179079 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 300/597] 10.222.0.59 () {56 vars in 1099 bytes} [Tue Oct 11 11:47:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:47:24.180548 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7215a6e4-2cec-47f7-ae6b-0b82985e4efa None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:24.201333 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 298/598] 10.222.0.59 () {56 vars in 1149 bytes} [Tue Oct 11 11:47:23 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:47:24.203778 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab9e4aa-236a-48e3-9670-f093e8505a3d None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:24.355747 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7215a6e4-2cec-47f7-ae6b-0b82985e4efa None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:24.375168 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:24.389482 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab9e4aa-236a-48e3-9670-f093e8505a3d None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:24.395196 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:24.437908 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf64717-d4da-4bae-8cc3-31f6c27ba5c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:24.440116 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf64717-d4da-4bae-8cc3-31f6c27ba5c6 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:24.441412 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf64717-d4da-4bae-8cc3-31f6c27ba5c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:24.442182 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf64717-d4da-4bae-8cc3-31f6c27ba5c6 None admin] PATH_INFO: `/v3/services` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:24.444457 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf64717-d4da-4bae-8cc3-31f6c27ba5c6 None admin] RBAC: Adding query filter params (type=network) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:24.445644 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf64717-d4da-4bae-8cc3-31f6c27ba5c6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:24.449880 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf64717-d4da-4bae-8cc3-31f6c27ba5c6 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:24.455969 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc5b898-d54e-4b83-863f-8c8c1eecbffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:24.457356 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc5b898-d54e-4b83-863f-8c8c1eecbffc None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:24.457840 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc5b898-d54e-4b83-863f-8c8c1eecbffc None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:24.458309 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc5b898-d54e-4b83-863f-8c8c1eecbffc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:24.467025 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 301/599] 10.222.0.59 () {56 vars in 1112 bytes} [Tue Oct 11 11:47:24 2022] GET /identity/v3/services?type=network => generated 350 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:47:24.467025 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf64717-d4da-4bae-8cc3-31f6c27ba5c6 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:24.484918 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc5b898-d54e-4b83-863f-8c8c1eecbffc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:24.487857 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc5b898-d54e-4b83-863f-8c8c1eecbffc None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:24.514504 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application [None req-ddc5b898-d54e-4b83-863f-8c8c1eecbffc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 11:47:24.514504 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:47:24.514504 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:47:24.514504 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:47:24.514504 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:47:24.514504 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:47:24.514504 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:47:24.514504 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:47:24.514504 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:47:24.514504 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:47:24.514504 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:47:24.514504 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:47:24.514504 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 11:47:24.514504 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 11:47:24.514504 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 11:47:24.514504 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 11:47:24.516470 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:47:24.516470 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:47:24.516470 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:47:24.516470 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:47:24.516470 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:47:24.516470 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:47:24.516470 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:47:24.516470 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application with Lock( Oct 11 11:47:24.516470 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:47:24.516470 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:47:24.516470 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:47:24.516470 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:47:24.516470 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:47:24.516470 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:47:24.516470 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:47:24.516470 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:47:24.516470 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 11:47:24.518307 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 11:47:24.518307 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 11:47:24.518307 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:47:24.518307 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 299/600] 10.222.0.59 () {56 vars in 1108 bytes} [Tue Oct 11 11:47:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:47:24.518307 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc5b898-d54e-4b83-863f-8c8c1eecbffc None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:24.685514 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf64717-d4da-4bae-8cc3-31f6c27ba5c6 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:24.754004 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc5b898-d54e-4b83-863f-8c8c1eecbffc None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:24.761663 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2589b6-5eb2-4cba-b894-3d9aceedb410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:24.762828 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2589b6-5eb2-4cba-b894-3d9aceedb410 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:24.763328 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2589b6-5eb2-4cba-b894-3d9aceedb410 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:24.763960 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2589b6-5eb2-4cba-b894-3d9aceedb410 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:24.806830 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:24.845994 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d57ff7-bbe0-415b-a08c-325bb09082ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:24.847816 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d57ff7-bbe0-415b-a08c-325bb09082ae None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:24.848639 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d57ff7-bbe0-415b-a08c-325bb09082ae None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:24.849396 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d57ff7-bbe0-415b-a08c-325bb09082ae None admin] PATH_INFO: `/v3/endpoints` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:24.851064 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d57ff7-bbe0-415b-a08c-325bb09082ae None admin] RBAC: Adding query filter params (interface=public, service_id=f8daeccbb5d948de8827dcde011fcc76, region_id=RegionOne) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:24.852086 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d57ff7-bbe0-415b-a08c-325bb09082ae None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:24.856174 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d57ff7-bbe0-415b-a08c-325bb09082ae None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:24.876163 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 300/601] 10.222.0.59 () {56 vars in 1251 bytes} [Tue Oct 11 11:47:24 2022] GET /identity/v3/endpoints?service_id=f8daeccbb5d948de8827dcde011fcc76&interface=public®ion_id=RegionOne => generated 194 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:47:24.876163 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d57ff7-bbe0-415b-a08c-325bb09082ae None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:24.924461 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2589b6-5eb2-4cba-b894-3d9aceedb410 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:24.931204 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:24.931502 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2589b6-5eb2-4cba-b894-3d9aceedb410 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:25.138597 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d57ff7-bbe0-415b-a08c-325bb09082ae None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:25.139598 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 302/602] 10.222.0.59 () {58 vars in 1282 bytes} [Tue Oct 11 11:47:24 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 676 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 11 11:47:25.141150 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2589b6-5eb2-4cba-b894-3d9aceedb410 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:25.214962 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b1df76-e0a5-474f-ba7d-b82d0bdecc86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:25.215267 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b1df76-e0a5-474f-ba7d-b82d0bdecc86 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:25.215569 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b1df76-e0a5-474f-ba7d-b82d0bdecc86 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:25.215988 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b1df76-e0a5-474f-ba7d-b82d0bdecc86 None admin] PATH_INFO: `/v3/domains` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:25.217210 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b1df76-e0a5-474f-ba7d-b82d0bdecc86 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:25.217685 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b1df76-e0a5-474f-ba7d-b82d0bdecc86 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:25.220725 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b1df76-e0a5-474f-ba7d-b82d0bdecc86 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:25.250006 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 301/603] 10.222.0.59 () {56 vars in 1109 bytes} [Tue Oct 11 11:47:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 370 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:47:25.250006 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b1df76-e0a5-474f-ba7d-b82d0bdecc86 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:25.256450 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:25.369204 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:25.377249 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2589b6-5eb2-4cba-b894-3d9aceedb410 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:25.396097 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b1df76-e0a5-474f-ba7d-b82d0bdecc86 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:25.445631 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee6616f-c214-45de-b2c1-70743132b912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:25.446411 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee6616f-c214-45de-b2c1-70743132b912 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:25.446790 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee6616f-c214-45de-b2c1-70743132b912 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:25.447131 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee6616f-c214-45de-b2c1-70743132b912 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:25.458029 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bcb28d-7e94-413e-b838-955506524762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:25.459361 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee6616f-c214-45de-b2c1-70743132b912 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:25.459537 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bcb28d-7e94-413e-b838-955506524762 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:25.459876 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bcb28d-7e94-413e-b838-955506524762 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:25.460416 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bcb28d-7e94-413e-b838-955506524762 None admin] PATH_INFO: `/v3/services` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:25.461292 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee6616f-c214-45de-b2c1-70743132b912 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:25.461560 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bcb28d-7e94-413e-b838-955506524762 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:25.465305 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bcb28d-7e94-413e-b838-955506524762 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:25.478006 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 303/604] 10.222.0.59 () {56 vars in 1087 bytes} [Tue Oct 11 11:47:25 2022] GET /identity/v3/services => generated 772 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:47:25.479492 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bcb28d-7e94-413e-b838-955506524762 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:25.479940 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application [None req-9ee6616f-c214-45de-b2c1-70743132b912 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 11:47:25.479940 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:47:25.479940 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:47:25.479940 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:47:25.479940 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:47:25.479940 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:47:25.479940 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:47:25.479940 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:47:25.479940 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:47:25.479940 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:47:25.479940 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:47:25.479940 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:47:25.479940 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 11:47:25.479940 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 11:47:25.479940 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 11:47:25.479940 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 11:47:25.481887 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:47:25.481887 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:47:25.481887 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:47:25.481887 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:47:25.481887 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:47:25.481887 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:47:25.481887 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:47:25.481887 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application with Lock( Oct 11 11:47:25.481887 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:47:25.481887 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:47:25.481887 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:47:25.481887 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:47:25.481887 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:47:25.481887 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:47:25.481887 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:47:25.481887 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:47:25.481887 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 11:47:25.483822 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 11:47:25.483822 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 11:47:25.483822 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:47:25.490597 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 302/605] 10.222.0.59 () {56 vars in 1108 bytes} [Tue Oct 11 11:47:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:47:25.490597 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee6616f-c214-45de-b2c1-70743132b912 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:25.707026 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bcb28d-7e94-413e-b838-955506524762 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:25.733870 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:25.766135 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee6616f-c214-45de-b2c1-70743132b912 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:25.771544 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81195c38-9edf-4894-9013-81012943b9a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:25.772520 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81195c38-9edf-4894-9013-81012943b9a7 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:25.772947 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81195c38-9edf-4894-9013-81012943b9a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:25.773373 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81195c38-9edf-4894-9013-81012943b9a7 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:25.788759 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81195c38-9edf-4894-9013-81012943b9a7 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:25.791273 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81195c38-9edf-4894-9013-81012943b9a7 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:25.814314 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application [None req-81195c38-9edf-4894-9013-81012943b9a7 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 11 11:47:25.814314 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:47:25.814314 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:47:25.814314 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:47:25.814314 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:47:25.814314 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:47:25.814314 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:47:25.814314 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:47:25.814314 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:47:25.814314 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:47:25.814314 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:47:25.814314 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:47:25.814314 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 11 11:47:25.814314 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 11 11:47:25.814314 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 11 11:47:25.814314 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 11 11:47:25.818834 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:47:25.818834 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:47:25.818834 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 11 11:47:25.818834 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 11:47:25.818834 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 11 11:47:25.818834 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 11:47:25.818834 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:47:25.818834 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:47:25.818834 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:47:25.818834 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:47:25.818834 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:47:25.818834 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application with Lock( Oct 11 11:47:25.818834 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:47:25.818834 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:47:25.818834 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:47:25.818834 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:47:25.818834 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:47:25.818834 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:47:25.822181 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:47:25.822181 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:47:25.822181 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 11 11:47:25.822181 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 11 11:47:25.822181 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 11 11:47:25.822181 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 11 11:47:25.822181 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 11 11:47:25.822181 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 11 11:47:25.822181 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 11 11:47:25.822181 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:47:25.824225 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 304/606] 10.222.0.59 () {56 vars in 1096 bytes} [Tue Oct 11 11:47:25 2022] GET /identity/v3/users/swift => generated 83 bytes in 345 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:47:25.826846 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81195c38-9edf-4894-9013-81012943b9a7 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:25.871525 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:25.886301 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b50e60-26d3-44a2-887e-f2d90482c06d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:25.887350 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b50e60-26d3-44a2-887e-f2d90482c06d None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:25.887804 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b50e60-26d3-44a2-887e-f2d90482c06d None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:25.888230 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b50e60-26d3-44a2-887e-f2d90482c06d None admin] PATH_INFO: `/v3/domains` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:25.889308 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b50e60-26d3-44a2-887e-f2d90482c06d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:25.889792 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b50e60-26d3-44a2-887e-f2d90482c06d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:25.892442 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b50e60-26d3-44a2-887e-f2d90482c06d None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:25.920432 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 303/607] 10.222.0.59 () {56 vars in 1109 bytes} [Tue Oct 11 11:47:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 431 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 11:47:25.934038 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b50e60-26d3-44a2-887e-f2d90482c06d None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:26.013759 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81195c38-9edf-4894-9013-81012943b9a7 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:26.078392 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabe1994-92fa-4919-9b3f-11058f658708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:26.079771 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabe1994-92fa-4919-9b3f-11058f658708 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:26.080232 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabe1994-92fa-4919-9b3f-11058f658708 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:26.080665 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabe1994-92fa-4919-9b3f-11058f658708 None admin] PATH_INFO: `/v3/users` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:26.081884 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabe1994-92fa-4919-9b3f-11058f658708 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:26.082384 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabe1994-92fa-4919-9b3f-11058f658708 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:26.084980 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabe1994-92fa-4919-9b3f-11058f658708 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:26.117006 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 305/608] 10.222.0.59 () {56 vars in 1135 bytes} [Tue Oct 11 11:47:25 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 11:47:26.127981 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabe1994-92fa-4919-9b3f-11058f658708 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:26.139631 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b50e60-26d3-44a2-887e-f2d90482c06d None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:26.184110 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:26.205831 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2746fb-bece-49da-bdcc-edf2e0a0e5dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:26.206863 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2746fb-bece-49da-bdcc-edf2e0a0e5dc None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:26.207366 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2746fb-bece-49da-bdcc-edf2e0a0e5dc None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:26.207813 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2746fb-bece-49da-bdcc-edf2e0a0e5dc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:26.220286 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2746fb-bece-49da-bdcc-edf2e0a0e5dc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:26.222961 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2746fb-bece-49da-bdcc-edf2e0a0e5dc None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:26.244792 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application [None req-ea2746fb-bece-49da-bdcc-edf2e0a0e5dc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 11:47:26.244792 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:47:26.244792 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:47:26.244792 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:47:26.244792 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:47:26.244792 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:47:26.244792 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:47:26.244792 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:47:26.244792 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:47:26.244792 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:47:26.244792 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:47:26.244792 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:47:26.244792 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 11 11:47:26.244792 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 11 11:47:26.244792 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 11 11:47:26.246515 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 11 11:47:26.246515 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:47:26.246515 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:47:26.246515 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:47:26.246515 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:47:26.246515 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:47:26.246515 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:47:26.246515 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:47:26.246515 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application with Lock( Oct 11 11:47:26.246515 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:47:26.246515 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:47:26.246515 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:47:26.246515 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:47:26.246515 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:47:26.246515 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:47:26.246515 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:47:26.246515 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:47:26.248179 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 11 11:47:26.248179 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 11 11:47:26.248179 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 11 11:47:26.248179 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 11 11:47:26.248179 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 11 11:47:26.248179 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 11 11:47:26.248179 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 11:47:26.248179 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:47:26.253253 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 304/609] 10.222.0.59 () {56 vars in 1111 bytes} [Tue Oct 11 11:47:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:47:26.256871 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2746fb-bece-49da-bdcc-edf2e0a0e5dc None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:26.306207 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabe1994-92fa-4919-9b3f-11058f658708 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:26.350925 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:26.367875 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d6b9c5-d6df-4bf2-9d08-8b4d3117c1a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:26.368860 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d6b9c5-d6df-4bf2-9d08-8b4d3117c1a3 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:26.369329 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d6b9c5-d6df-4bf2-9d08-8b4d3117c1a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:26.369794 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d6b9c5-d6df-4bf2-9d08-8b4d3117c1a3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:26.444876 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2746fb-bece-49da-bdcc-edf2e0a0e5dc None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:26.491879 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4d2d9b-1f8b-41e3-b81d-0f6567a08128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:26.492506 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4d2d9b-1f8b-41e3-b81d-0f6567a08128 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:26.492783 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4d2d9b-1f8b-41e3-b81d-0f6567a08128 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:26.493063 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4d2d9b-1f8b-41e3-b81d-0f6567a08128 None admin] PATH_INFO: `/v3/projects` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:26.493809 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4d2d9b-1f8b-41e3-b81d-0f6567a08128 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:26.494114 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4d2d9b-1f8b-41e3-b81d-0f6567a08128 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:26.494967 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d6b9c5-d6df-4bf2-9d08-8b4d3117c1a3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:26.495728 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4d2d9b-1f8b-41e3-b81d-0f6567a08128 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:26.497838 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d6b9c5-d6df-4bf2-9d08-8b4d3117c1a3 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:26.514338 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 305/610] 10.222.0.59 () {56 vars in 1148 bytes} [Tue Oct 11 11:47:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:47:26.515825 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4d2d9b-1f8b-41e3-b81d-0f6567a08128 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:26.575332 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:26.631793 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4d2d9b-1f8b-41e3-b81d-0f6567a08128 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:26.644087 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:26.671038 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7a20ee-5185-4e34-a848-0fd192ed5b63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:26.671820 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7a20ee-5185-4e34-a848-0fd192ed5b63 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:26.672193 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7a20ee-5185-4e34-a848-0fd192ed5b63 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:26.672559 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7a20ee-5185-4e34-a848-0fd192ed5b63 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:26.673544 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7a20ee-5185-4e34-a848-0fd192ed5b63 None admin] RBAC: Adding query filter params (role.id=f5a43d8f3fa24a87a7abe7492cf6f87f, scope.project.id=26ecf552cc9a4e5ca1d380509683dc4a, user.id=8a19eceb62ce4157a4c5fe1866a89df9) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:26.673948 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7a20ee-5185-4e34-a848-0fd192ed5b63 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:26.675532 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7a20ee-5185-4e34-a848-0fd192ed5b63 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:26.686732 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 306/611] 10.222.0.59 () {56 vars in 1374 bytes} [Tue Oct 11 11:47:26 2022] GET /identity/v3/role_assignments?user.id=8a19eceb62ce4157a4c5fe1866a89df9&scope.project.id=26ecf552cc9a4e5ca1d380509683dc4a&role.id=f5a43d8f3fa24a87a7abe7492cf6f87f => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:47:26.767528 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 306/612] 10.222.0.59 () {58 vars in 1282 bytes} [Tue Oct 11 11:47:26 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 641 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 11 11:47:26.777255 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7a20ee-5185-4e34-a848-0fd192ed5b63 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:26.857505 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92d7422-9a55-4e8f-a838-3c0b223e2b44 None None] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:26.857742 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92d7422-9a55-4e8f-a838-3c0b223e2b44 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:26.857965 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92d7422-9a55-4e8f-a838-3c0b223e2b44 None None] PATH_INFO: `/` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:26.859717 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 307/613] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:47:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 11 11:47:26.865579 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec95f771-aecd-4c69-b7db-fa21e3dc543e None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:26.865865 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec95f771-aecd-4c69-b7db-fa21e3dc543e None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:26.866132 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec95f771-aecd-4c69-b7db-fa21e3dc543e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:26.893292 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-ec95f771-aecd-4c69-b7db-fa21e3dc543e None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:47:26.894537 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-ec95f771-aecd-4c69-b7db-fa21e3dc543e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:47:27.033308 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7a20ee-5185-4e34-a848-0fd192ed5b63 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:27.104700 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 308/614] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:47:26 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 11:47:27.121285 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae9d749-66ff-4b7c-b0ea-ed8bc598dcba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:27.122874 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae9d749-66ff-4b7c-b0ea-ed8bc598dcba None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:27.123713 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae9d749-66ff-4b7c-b0ea-ed8bc598dcba None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:27.124473 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae9d749-66ff-4b7c-b0ea-ed8bc598dcba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:27.145239 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae9d749-66ff-4b7c-b0ea-ed8bc598dcba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:27.150655 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae9d749-66ff-4b7c-b0ea-ed8bc598dcba None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:27.153209 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc739625-4547-4fd7-aada-bc8e919f94c6 None None] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:27.153209 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc739625-4547-4fd7-aada-bc8e919f94c6 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:27.153644 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc739625-4547-4fd7-aada-bc8e919f94c6 None None] PATH_INFO: `/` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:27.157588 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 309/615] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:47:27 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 11 11:47:27.166800 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d401d9f-7bec-465b-b4eb-469b162c0341 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:27.167248 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d401d9f-7bec-465b-b4eb-469b162c0341 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:27.167709 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d401d9f-7bec-465b-b4eb-469b162c0341 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:27.183771 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application [None req-0ae9d749-66ff-4b7c-b0ea-ed8bc598dcba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 11:47:27.183771 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:47:27.183771 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:47:27.183771 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:47:27.183771 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:47:27.183771 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:47:27.183771 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:47:27.183771 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:47:27.183771 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:47:27.183771 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:47:27.183771 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:47:27.183771 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:47:27.183771 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 11:47:27.183771 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 11:47:27.183771 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 11:47:27.183771 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 11:47:27.185564 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:47:27.185564 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:47:27.185564 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:47:27.185564 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:47:27.185564 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:47:27.185564 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:47:27.185564 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:47:27.185564 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application with Lock( Oct 11 11:47:27.185564 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:47:27.185564 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:47:27.185564 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:47:27.185564 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:47:27.185564 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:47:27.185564 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:47:27.185564 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:47:27.185564 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:47:27.185564 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 11:47:27.187298 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 11:47:27.187298 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 11:47:27.187298 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:47:27.189076 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 307/616] 10.222.0.59 () {56 vars in 1108 bytes} [Tue Oct 11 11:47:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 413 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:47:27.198213 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae9d749-66ff-4b7c-b0ea-ed8bc598dcba None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:27.206261 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-8d401d9f-7bec-465b-b4eb-469b162c0341 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:47:27.208369 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-8d401d9f-7bec-465b-b4eb-469b162c0341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:47:27.285813 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:27.410249 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:27.441582 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 310/617] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:47:27 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 11 11:47:27.442249 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae9d749-66ff-4b7c-b0ea-ed8bc598dcba None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:27.447223 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d401d9f-7bec-465b-b4eb-469b162c0341 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:27.516917 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbe2ca8-a027-444c-ad84-ff93de09296e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:27.517987 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbe2ca8-a027-444c-ad84-ff93de09296e None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:27.518478 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbe2ca8-a027-444c-ad84-ff93de09296e None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:27.518983 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbe2ca8-a027-444c-ad84-ff93de09296e None admin] PATH_INFO: `/v3/domains` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:27.520156 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbe2ca8-a027-444c-ad84-ff93de09296e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:27.520724 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbe2ca8-a027-444c-ad84-ff93de09296e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:27.523505 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbe2ca8-a027-444c-ad84-ff93de09296e None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:27.556847 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 308/618] 10.222.0.59 () {56 vars in 1109 bytes} [Tue Oct 11 11:47:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:47:27.566966 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbe2ca8-a027-444c-ad84-ff93de09296e None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:27.698487 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d401d9f-7bec-465b-b4eb-469b162c0341 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:27.741569 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:27.774671 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbe2ca8-a027-444c-ad84-ff93de09296e None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:27.816970 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04536b5f-9cf5-4e00-9c70-efe58b95f825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:27.818062 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04536b5f-9cf5-4e00-9c70-efe58b95f825 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:27.819101 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04536b5f-9cf5-4e00-9c70-efe58b95f825 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:27.819577 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04536b5f-9cf5-4e00-9c70-efe58b95f825 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:27.820552 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04536b5f-9cf5-4e00-9c70-efe58b95f825 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:27.822249 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04536b5f-9cf5-4e00-9c70-efe58b95f825 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:27.836055 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application [None req-04536b5f-9cf5-4e00-9c70-efe58b95f825 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 11 11:47:27.836055 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:47:27.836055 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 11 11:47:27.836055 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 11 11:47:27.836055 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 11 11:47:27.836055 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 11 11:47:27.836055 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 11 11:47:27.836055 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 11:47:27.836055 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 11 11:47:27.836055 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:47:27.836055 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 11 11:47:27.836055 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:47:27.836055 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:47:27.836055 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:47:27.836055 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:47:27.836055 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:47:27.836055 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:47:27.837141 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:47:27.837141 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:47:27.837141 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:47:27.837141 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:47:27.837141 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:47:27.837141 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:47:27.837141 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 11 11:47:27.837141 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 11 11:47:27.837141 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 11 11:47:27.837141 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 11 11:47:27.837141 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:47:27.837141 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:47:27.837141 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:47:27.837141 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:47:27.837141 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:47:27.837141 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:47:27.838474 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:47:27.838474 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application with Lock( Oct 11 11:47:27.838474 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:47:27.838474 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:47:27.838474 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:47:27.838474 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:47:27.838474 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:47:27.838474 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:47:27.838474 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:47:27.838474 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:47:27.838474 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 11 11:47:27.838474 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 11:47:27.838474 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 11 11:47:27.838474 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:47:27.838474 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 311/619] 10.222.0.59 () {56 vars in 1129 bytes} [Tue Oct 11 11:47:27 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 395 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:47:27.839600 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04536b5f-9cf5-4e00-9c70-efe58b95f825 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:27.864235 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38152aa-37c9-482f-b58c-b0d13e104792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:27.865426 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38152aa-37c9-482f-b58c-b0d13e104792 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:27.865905 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38152aa-37c9-482f-b58c-b0d13e104792 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:27.866500 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38152aa-37c9-482f-b58c-b0d13e104792 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:27.880352 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38152aa-37c9-482f-b58c-b0d13e104792 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:27.883486 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38152aa-37c9-482f-b58c-b0d13e104792 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:27.905935 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application [None req-c38152aa-37c9-482f-b58c-b0d13e104792 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 11:47:27.905935 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:47:27.905935 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:47:27.905935 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:47:27.905935 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:47:27.905935 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:47:27.905935 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:47:27.905935 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:47:27.905935 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:47:27.905935 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:47:27.905935 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:47:27.905935 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:47:27.905935 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 11 11:47:27.905935 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 11 11:47:27.905935 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 11 11:47:27.906956 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 11 11:47:27.906956 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:47:27.906956 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:47:27.906956 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:47:27.906956 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:47:27.906956 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:47:27.906956 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:47:27.906956 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:47:27.906956 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application with Lock( Oct 11 11:47:27.906956 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:47:27.906956 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:47:27.906956 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:47:27.906956 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:47:27.906956 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:47:27.906956 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:47:27.906956 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:47:27.906956 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:47:27.907903 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 11 11:47:27.907903 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 11 11:47:27.907903 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 11 11:47:27.907903 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 11 11:47:27.907903 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 11 11:47:27.907903 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 11 11:47:27.907903 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 11:47:27.907903 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:47:27.908841 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 309/620] 10.222.0.59 () {56 vars in 1111 bytes} [Tue Oct 11 11:47:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 11 11:47:27.910480 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38152aa-37c9-482f-b58c-b0d13e104792 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:27.944198 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:28.035267 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04536b5f-9cf5-4e00-9c70-efe58b95f825 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:28.088423 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2442a0f-3635-4321-b606-c127eed08e17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:28.089629 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2442a0f-3635-4321-b606-c127eed08e17 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:28.090159 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2442a0f-3635-4321-b606-c127eed08e17 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:28.090764 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2442a0f-3635-4321-b606-c127eed08e17 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:28.091708 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2442a0f-3635-4321-b606-c127eed08e17 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:28.091816 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38152aa-37c9-482f-b58c-b0d13e104792 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:28.093814 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2442a0f-3635-4321-b606-c127eed08e17 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:28.106225 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application [None req-d2442a0f-3635-4321-b606-c127eed08e17 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 11 11:47:28.106225 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:47:28.106225 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 11 11:47:28.106225 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 11 11:47:28.106225 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 11 11:47:28.106225 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 11 11:47:28.106225 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 11 11:47:28.106225 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 11:47:28.106225 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 11 11:47:28.106225 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:47:28.106225 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 11 11:47:28.106225 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:47:28.106225 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:47:28.106225 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:47:28.106225 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:47:28.106225 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:47:28.107510 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:47:28.107510 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:47:28.107510 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:47:28.107510 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:47:28.107510 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:47:28.107510 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:47:28.107510 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:47:28.107510 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 11 11:47:28.107510 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 11 11:47:28.107510 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 11 11:47:28.107510 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 11 11:47:28.107510 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:47:28.107510 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:47:28.107510 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:47:28.107510 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:47:28.107510 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:47:28.107510 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:47:28.108744 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:47:28.108744 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application with Lock( Oct 11 11:47:28.108744 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:47:28.108744 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:47:28.108744 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:47:28.108744 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:47:28.108744 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:47:28.108744 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:47:28.108744 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:47:28.108744 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:47:28.108744 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 11 11:47:28.108744 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 11:47:28.108744 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 11 11:47:28.108744 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:47:28.117388 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 312/621] 10.222.0.59 () {56 vars in 1132 bytes} [Tue Oct 11 11:47:27 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:47:28.118774 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2442a0f-3635-4321-b606-c127eed08e17 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:28.130634 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811ae782-6bf2-469a-a9ab-912e18a9eb78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:28.131262 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811ae782-6bf2-469a-a9ab-912e18a9eb78 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:28.131533 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811ae782-6bf2-469a-a9ab-912e18a9eb78 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:28.131803 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811ae782-6bf2-469a-a9ab-912e18a9eb78 None admin] PATH_INFO: `/v3/services` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:28.132443 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811ae782-6bf2-469a-a9ab-912e18a9eb78 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:28.132742 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811ae782-6bf2-469a-a9ab-912e18a9eb78 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:28.134483 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811ae782-6bf2-469a-a9ab-912e18a9eb78 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:28.139875 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 310/622] 10.222.0.59 () {56 vars in 1124 bytes} [Tue Oct 11 11:47:27 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:47:28.140740 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811ae782-6bf2-469a-a9ab-912e18a9eb78 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:28.160130 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:28.257779 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2442a0f-3635-4321-b606-c127eed08e17 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:28.271937 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811ae782-6bf2-469a-a9ab-912e18a9eb78 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:28.285627 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:28.308055 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420fae90-da94-4f7c-aa35-9c974acaae19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:28.309267 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420fae90-da94-4f7c-aa35-9c974acaae19 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:28.309736 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420fae90-da94-4f7c-aa35-9c974acaae19 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:28.310411 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420fae90-da94-4f7c-aa35-9c974acaae19 None admin] PATH_INFO: `/v3/projects` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:28.311726 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420fae90-da94-4f7c-aa35-9c974acaae19 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:28.312483 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420fae90-da94-4f7c-aa35-9c974acaae19 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:28.314294 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420fae90-da94-4f7c-aa35-9c974acaae19 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:28.316620 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf178a96-4873-4681-b124-983b644fcac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:28.317692 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf178a96-4873-4681-b124-983b644fcac8 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:28.317939 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf178a96-4873-4681-b124-983b644fcac8 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:28.318306 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf178a96-4873-4681-b124-983b644fcac8 None admin] PATH_INFO: `/v3/services` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:28.319098 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf178a96-4873-4681-b124-983b644fcac8 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:28.319496 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf178a96-4873-4681-b124-983b644fcac8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:28.321160 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf178a96-4873-4681-b124-983b644fcac8 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:28.334285 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 311/624] 10.222.0.59 () {56 vars in 1126 bytes} [Tue Oct 11 11:47:28 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:47:28.334285 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf178a96-4873-4681-b124-983b644fcac8 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:28.334641 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 313/624] 10.222.0.59 () {56 vars in 1148 bytes} [Tue Oct 11 11:47:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:47:28.335746 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420fae90-da94-4f7c-aa35-9c974acaae19 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:28.441232 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:28.484200 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf178a96-4873-4681-b124-983b644fcac8 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:28.508537 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420fae90-da94-4f7c-aa35-9c974acaae19 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:28.528997 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310bd838-a207-4ddd-ac23-1c4619672942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:28.529792 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310bd838-a207-4ddd-ac23-1c4619672942 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:28.530164 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310bd838-a207-4ddd-ac23-1c4619672942 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:28.530572 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310bd838-a207-4ddd-ac23-1c4619672942 None admin] PATH_INFO: `/v3/services` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:28.531440 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310bd838-a207-4ddd-ac23-1c4619672942 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:28.531841 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310bd838-a207-4ddd-ac23-1c4619672942 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:28.533433 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310bd838-a207-4ddd-ac23-1c4619672942 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:28.542678 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 312/625] 10.222.0.59 () {56 vars in 1124 bytes} [Tue Oct 11 11:47:28 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:47:28.542678 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310bd838-a207-4ddd-ac23-1c4619672942 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:28.563461 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2885e8-72e4-43ca-9248-b66b70394111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:28.564657 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2885e8-72e4-43ca-9248-b66b70394111 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:28.565035 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2885e8-72e4-43ca-9248-b66b70394111 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:28.565495 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2885e8-72e4-43ca-9248-b66b70394111 None admin] PATH_INFO: `/v3/services` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:28.566259 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2885e8-72e4-43ca-9248-b66b70394111 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:28.566724 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2885e8-72e4-43ca-9248-b66b70394111 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:28.568303 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2885e8-72e4-43ca-9248-b66b70394111 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:28.582845 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 314/626] 10.222.0.59 () {56 vars in 1126 bytes} [Tue Oct 11 11:47:28 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:47:28.584056 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2885e8-72e4-43ca-9248-b66b70394111 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:28.712150 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:28.723010 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310bd838-a207-4ddd-ac23-1c4619672942 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:28.794257 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb531463-be21-448c-a2c9-09fccad79e24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:28.794934 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb531463-be21-448c-a2c9-09fccad79e24 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:28.795214 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb531463-be21-448c-a2c9-09fccad79e24 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:28.795602 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb531463-be21-448c-a2c9-09fccad79e24 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:28.796387 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb531463-be21-448c-a2c9-09fccad79e24 None admin] RBAC: Adding query filter params (role.id=5cf1f4bd2fef46ab9c6eff56d52f9f69, scope.project.id=26ecf552cc9a4e5ca1d380509683dc4a, user.id=76e7fee1993a40d5ba4295e7daeb3e29) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:28.796698 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb531463-be21-448c-a2c9-09fccad79e24 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:28.798822 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb531463-be21-448c-a2c9-09fccad79e24 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:28.808284 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 313/627] 10.222.0.59 () {56 vars in 1374 bytes} [Tue Oct 11 11:47:28 2022] GET /identity/v3/role_assignments?user.id=76e7fee1993a40d5ba4295e7daeb3e29&scope.project.id=26ecf552cc9a4e5ca1d380509683dc4a&role.id=5cf1f4bd2fef46ab9c6eff56d52f9f69 => generated 608 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:47:28.831295 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2885e8-72e4-43ca-9248-b66b70394111 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:28.875858 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3eecce1-d218-411c-81ac-aa752897c4a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:28.876568 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eecce1-d218-411c-81ac-aa752897c4a5 None admin] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:28.876848 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eecce1-d218-411c-81ac-aa752897c4a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:28.877119 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eecce1-d218-411c-81ac-aa752897c4a5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:28.877831 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3eecce1-d218-411c-81ac-aa752897c4a5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:28.879561 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3eecce1-d218-411c-81ac-aa752897c4a5 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:28.912075 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 315/628] 10.222.0.59 () {60 vars in 1144 bytes} [Tue Oct 11 11:47:28 2022] POST /identity/v3/endpoints => generated 346 bytes in 328 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 11:47:29.172027 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d36b968-38d9-4f31-b8ee-124c2f7137c9 None None] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:29.172442 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d36b968-38d9-4f31-b8ee-124c2f7137c9 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:29.172941 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d36b968-38d9-4f31-b8ee-124c2f7137c9 None None] PATH_INFO: `/` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:29.176199 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 314/629] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:47:29 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:47:29.182041 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dffa897-c271-4688-914b-233b5f57a2c4 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:29.182348 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dffa897-c271-4688-914b-233b5f57a2c4 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:29.182620 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dffa897-c271-4688-914b-233b5f57a2c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:29.207942 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-0dffa897-c271-4688-914b-233b5f57a2c4 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:47:29.209179 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-0dffa897-c271-4688-914b-233b5f57a2c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:47:29.348385 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 316/630] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:47:29 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 11 11:47:29.649390 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d36b968-38d9-4f31-b8ee-124c2f7137c9 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:29.652861 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:29.862309 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d36b968-38d9-4f31-b8ee-124c2f7137c9 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:29.877201 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748c766b-aee9-471e-93d5-1f1fc5e5fb1a None None] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:29.877813 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748c766b-aee9-471e-93d5-1f1fc5e5fb1a None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:29.878089 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748c766b-aee9-471e-93d5-1f1fc5e5fb1a None None] PATH_INFO: `/` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:29.884260 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 317/631] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:47:29 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:47:29.891265 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d894995b-f793-47c1-8f47-6a231dad513a None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:29.891930 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d894995b-f793-47c1-8f47-6a231dad513a None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:29.892402 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d894995b-f793-47c1-8f47-6a231dad513a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:29.907297 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464b852a-5df4-46e1-87c5-3d8f476f6261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:29.908457 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464b852a-5df4-46e1-87c5-3d8f476f6261 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:29.908930 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464b852a-5df4-46e1-87c5-3d8f476f6261 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:29.909731 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464b852a-5df4-46e1-87c5-3d8f476f6261 None admin] PATH_INFO: `/v3/services/network` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:29.910521 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464b852a-5df4-46e1-87c5-3d8f476f6261 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:29.912262 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464b852a-5df4-46e1-87c5-3d8f476f6261 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:29.928579 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application [None req-464b852a-5df4-46e1-87c5-3d8f476f6261 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 11 11:47:29.928579 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:47:29.928579 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 11 11:47:29.928579 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 11 11:47:29.928579 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 11 11:47:29.928579 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 11 11:47:29.928579 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 11 11:47:29.928579 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 11:47:29.928579 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 11 11:47:29.928579 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:47:29.928579 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 11 11:47:29.928579 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:47:29.928579 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:47:29.928579 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:47:29.928579 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:47:29.928579 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:47:29.928579 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:47:29.929634 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:47:29.929634 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:47:29.929634 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:47:29.929634 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:47:29.929634 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:47:29.929634 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:47:29.929634 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 11 11:47:29.929634 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 11 11:47:29.929634 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 11 11:47:29.929634 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 11 11:47:29.929634 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:47:29.929634 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:47:29.929634 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:47:29.929634 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:47:29.929634 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:47:29.929634 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:47:29.930456 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:47:29.930456 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application with Lock( Oct 11 11:47:29.930456 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:47:29.930456 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:47:29.930456 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:47:29.930456 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:47:29.930456 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:47:29.930456 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:47:29.930456 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:47:29.930456 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:47:29.930456 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 11 11:47:29.930456 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 11:47:29.930456 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 11 11:47:29.930456 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:47:29.937133 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-d894995b-f793-47c1-8f47-6a231dad513a None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:47:29.941310 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 315/632] 10.222.0.59 () {56 vars in 1111 bytes} [Tue Oct 11 11:47:29 2022] GET /identity/v3/services/network => generated 88 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:47:29.943190 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-d894995b-f793-47c1-8f47-6a231dad513a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:47:29.949860 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464b852a-5df4-46e1-87c5-3d8f476f6261 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:30.015010 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:30.137294 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 318/633] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:47:29 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 11:47:30.149436 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:30.189595 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464b852a-5df4-46e1-87c5-3d8f476f6261 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:30.226203 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b7c636-9127-4aca-8cc2-0fb09b91a778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:30.226863 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b7c636-9127-4aca-8cc2-0fb09b91a778 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:30.227141 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b7c636-9127-4aca-8cc2-0fb09b91a778 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:30.227407 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b7c636-9127-4aca-8cc2-0fb09b91a778 None admin] PATH_INFO: `/v3/services` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:30.228045 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b7c636-9127-4aca-8cc2-0fb09b91a778 None admin] RBAC: Adding query filter params (name=network) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:30.228345 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b7c636-9127-4aca-8cc2-0fb09b91a778 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:30.229738 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b7c636-9127-4aca-8cc2-0fb09b91a778 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:30.239985 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 316/634] 10.222.0.59 () {56 vars in 1112 bytes} [Tue Oct 11 11:47:29 2022] GET /identity/v3/services?name=network => generated 124 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 11:47:30.241440 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d894995b-f793-47c1-8f47-6a231dad513a None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:30.379644 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d894995b-f793-47c1-8f47-6a231dad513a None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:30.427931 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8ebcb3-631d-476f-b7b9-a00859e16c3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:30.428559 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8ebcb3-631d-476f-b7b9-a00859e16c3e None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:30.428834 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8ebcb3-631d-476f-b7b9-a00859e16c3e None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:30.429100 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8ebcb3-631d-476f-b7b9-a00859e16c3e None admin] PATH_INFO: `/v3/services` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:30.429738 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8ebcb3-631d-476f-b7b9-a00859e16c3e None admin] RBAC: Adding query filter params (type=network) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:30.430034 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8ebcb3-631d-476f-b7b9-a00859e16c3e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:30.431472 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8ebcb3-631d-476f-b7b9-a00859e16c3e None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:30.442104 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 319/635] 10.222.0.59 () {56 vars in 1112 bytes} [Tue Oct 11 11:47:30 2022] GET /identity/v3/services?type=network => generated 350 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:47:30.443538 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b7c636-9127-4aca-8cc2-0fb09b91a778 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:30.526243 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8ebcb3-631d-476f-b7b9-a00859e16c3e None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:30.568302 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:30.592458 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b7c636-9127-4aca-8cc2-0fb09b91a778 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:30.659012 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:30.669025 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3573941b-d7ba-43e6-9955-5c2d99dfcf22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:30.670055 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3573941b-d7ba-43e6-9955-5c2d99dfcf22 None admin] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:30.670585 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3573941b-d7ba-43e6-9955-5c2d99dfcf22 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:30.671165 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3573941b-d7ba-43e6-9955-5c2d99dfcf22 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:30.672225 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3573941b-d7ba-43e6-9955-5c2d99dfcf22 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:30.673990 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3573941b-d7ba-43e6-9955-5c2d99dfcf22 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:30.704748 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 317/636] 10.222.0.59 () {60 vars in 1144 bytes} [Tue Oct 11 11:47:30 2022] POST /identity/v3/endpoints => generated 337 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 11:47:30.727547 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8ebcb3-631d-476f-b7b9-a00859e16c3e None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:30.791660 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220d9cc0-02dd-44e6-9218-a1f6b79dc3ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:30.792612 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220d9cc0-02dd-44e6-9218-a1f6b79dc3ee None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:30.793028 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220d9cc0-02dd-44e6-9218-a1f6b79dc3ee None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:30.793436 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220d9cc0-02dd-44e6-9218-a1f6b79dc3ee None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:30.831007 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220d9cc0-02dd-44e6-9218-a1f6b79dc3ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 11:47:30.841692 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application [None req-220d9cc0-02dd-44e6-9218-a1f6b79dc3ee None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 11 11:47:30.841692 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:47:30.841692 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:47:30.841692 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:47:30.841692 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:47:30.841692 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:47:30.841692 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:47:30.841692 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:47:30.841692 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:47:30.841692 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:47:30.841692 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:47:30.841692 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:47:30.841692 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 11:47:30.841692 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 11:47:30.841692 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 11:47:30.841692 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 11:47:30.842789 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 11:47:30.842789 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 11:47:30.842789 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 11:47:30.842789 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 11:47:30.842789 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:47:30.842789 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:47:30.842789 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:47:30.842789 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:47:30.842789 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:47:30.842789 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:47:30.842789 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:47:30.842789 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application with Lock( Oct 11 11:47:30.842789 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:47:30.842789 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:47:30.842789 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:47:30.842789 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:47:30.843706 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:47:30.843706 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:47:30.843706 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:47:30.843706 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:47:30.843706 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 11:47:30.843706 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 11:47:30.843706 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 11:47:30.843706 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 11:47:30.843706 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 11:47:30.843706 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 11:47:30.843706 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 11 11:47:30.843706 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:47:30.843706 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 320/637] 10.222.0.59 () {56 vars in 1120 bytes} [Tue Oct 11 11:47:30 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:47:30.849940 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3573941b-d7ba-43e6-9955-5c2d99dfcf22 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:30.995881 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3573941b-d7ba-43e6-9955-5c2d99dfcf22 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:31.033838 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571ac153-48e3-46be-8f26-821364e9f68f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:31.035249 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571ac153-48e3-46be-8f26-821364e9f68f None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:31.035734 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571ac153-48e3-46be-8f26-821364e9f68f None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:31.036202 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571ac153-48e3-46be-8f26-821364e9f68f None admin] PATH_INFO: `/v3/roles` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:31.037075 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571ac153-48e3-46be-8f26-821364e9f68f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:31.037580 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571ac153-48e3-46be-8f26-821364e9f68f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:31.039281 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571ac153-48e3-46be-8f26-821364e9f68f None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:31.049404 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 318/638] 10.222.0.59 () {56 vars in 1115 bytes} [Tue Oct 11 11:47:30 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:47:31.058342 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220d9cc0-02dd-44e6-9218-a1f6b79dc3ee None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:31.149667 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:31.175143 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220d9cc0-02dd-44e6-9218-a1f6b79dc3ee None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:31.216923 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2b38fc-dfdd-4fb2-b074-d082ff2b3021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:31.217488 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2b38fc-dfdd-4fb2-b074-d082ff2b3021 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:31.217762 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2b38fc-dfdd-4fb2-b074-d082ff2b3021 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:31.218031 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2b38fc-dfdd-4fb2-b074-d082ff2b3021 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:31.231135 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2b38fc-dfdd-4fb2-b074-d082ff2b3021 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:31.232630 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2b38fc-dfdd-4fb2-b074-d082ff2b3021 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:31.249194 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application [None req-5b2b38fc-dfdd-4fb2-b074-d082ff2b3021 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 11:47:31.249194 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:47:31.249194 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:47:31.249194 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:47:31.249194 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:47:31.249194 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:47:31.249194 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:47:31.249194 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:47:31.249194 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:47:31.249194 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:47:31.249194 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:47:31.249194 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:47:31.249194 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 11:47:31.249194 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 11:47:31.249194 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 11:47:31.249194 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 11:47:31.250193 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:47:31.250193 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:47:31.250193 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:47:31.250193 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:47:31.250193 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:47:31.250193 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:47:31.250193 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:47:31.250193 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application with Lock( Oct 11 11:47:31.250193 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:47:31.250193 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:47:31.250193 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:47:31.250193 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:47:31.250193 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:47:31.250193 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:47:31.250193 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:47:31.250193 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:47:31.250193 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 11:47:31.251189 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 11:47:31.251189 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 11:47:31.251189 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:47:31.254836 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 321/639] 10.222.0.59 () {56 vars in 1108 bytes} [Tue Oct 11 11:47:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:47:31.259759 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571ac153-48e3-46be-8f26-821364e9f68f None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:31.323829 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:31.389909 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571ac153-48e3-46be-8f26-821364e9f68f None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:31.430846 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648966d9-aaf4-4dd1-970d-992cc89a76af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:31.431477 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648966d9-aaf4-4dd1-970d-992cc89a76af None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:31.431760 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648966d9-aaf4-4dd1-970d-992cc89a76af None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:31.432034 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648966d9-aaf4-4dd1-970d-992cc89a76af None admin] PATH_INFO: `/v3/domains` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:31.432704 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648966d9-aaf4-4dd1-970d-992cc89a76af None admin] RBAC: Adding query filter params (name=Default) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:31.433004 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648966d9-aaf4-4dd1-970d-992cc89a76af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:31.434408 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648966d9-aaf4-4dd1-970d-992cc89a76af None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:31.448580 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 319/640] 10.222.0.59 () {56 vars in 1109 bytes} [Tue Oct 11 11:47:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:47:31.454029 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2b38fc-dfdd-4fb2-b074-d082ff2b3021 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:31.569846 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2b38fc-dfdd-4fb2-b074-d082ff2b3021 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:31.603010 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:31.612595 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f20bfa-e5d1-43d7-a6ef-b43d03f487bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:31.613290 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f20bfa-e5d1-43d7-a6ef-b43d03f487bf None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:31.613568 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f20bfa-e5d1-43d7-a6ef-b43d03f487bf None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:31.613848 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f20bfa-e5d1-43d7-a6ef-b43d03f487bf None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:31.623252 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f20bfa-e5d1-43d7-a6ef-b43d03f487bf None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:31.624772 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f20bfa-e5d1-43d7-a6ef-b43d03f487bf None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:31.637812 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application [None req-a9f20bfa-e5d1-43d7-a6ef-b43d03f487bf None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 11 11:47:31.637812 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:47:31.637812 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:47:31.637812 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:47:31.637812 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:47:31.637812 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:47:31.637812 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:47:31.637812 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:47:31.637812 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:47:31.637812 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:47:31.637812 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:47:31.637812 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:47:31.637812 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 11 11:47:31.637812 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 11 11:47:31.637812 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 11 11:47:31.637812 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 11 11:47:31.638948 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:47:31.638948 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:47:31.638948 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 11 11:47:31.638948 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 11:47:31.638948 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 11 11:47:31.638948 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 11:47:31.638948 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:47:31.638948 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:47:31.638948 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:47:31.638948 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:47:31.638948 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:47:31.638948 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application with Lock( Oct 11 11:47:31.638948 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:47:31.638948 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:47:31.638948 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:47:31.638948 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:47:31.638948 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:47:31.638948 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:47:31.639877 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:47:31.639877 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:47:31.639877 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 11 11:47:31.639877 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 11 11:47:31.639877 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 11 11:47:31.639877 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 11 11:47:31.639877 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 11 11:47:31.639877 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 11 11:47:31.639877 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 11 11:47:31.639877 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:47:31.639877 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 322/641] 10.222.0.59 () {56 vars in 1117 bytes} [Tue Oct 11 11:47:31 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:47:31.645704 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648966d9-aaf4-4dd1-970d-992cc89a76af None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:31.761158 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648966d9-aaf4-4dd1-970d-992cc89a76af None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:31.772839 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:31.809085 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631297a5-4c71-4960-9fe5-c1ec2bc474e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:31.809741 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631297a5-4c71-4960-9fe5-c1ec2bc474e4 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:31.810025 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631297a5-4c71-4960-9fe5-c1ec2bc474e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:31.810302 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631297a5-4c71-4960-9fe5-c1ec2bc474e4 None admin] PATH_INFO: `/v3/users` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:31.811114 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631297a5-4c71-4960-9fe5-c1ec2bc474e4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:31.811425 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631297a5-4c71-4960-9fe5-c1ec2bc474e4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:31.812955 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631297a5-4c71-4960-9fe5-c1ec2bc474e4 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:31.823521 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d11e5c-dc5a-4341-ba8e-3542c923c4a1 None None] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:31.823864 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d11e5c-dc5a-4341-ba8e-3542c923c4a1 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:31.824234 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d11e5c-dc5a-4341-ba8e-3542c923c4a1 None None] PATH_INFO: `/` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:31.826046 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 323/642] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:47:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:47:31.832096 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e03cd1-8e2e-48ce-9a73-0bad719b94c8 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:31.832516 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e03cd1-8e2e-48ce-9a73-0bad719b94c8 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:31.832887 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e03cd1-8e2e-48ce-9a73-0bad719b94c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:31.834904 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 320/643] 10.222.0.59 () {56 vars in 1149 bytes} [Tue Oct 11 11:47:31 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:47:31.841614 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631297a5-4c71-4960-9fe5-c1ec2bc474e4 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:31.856020 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-10e03cd1-8e2e-48ce-9a73-0bad719b94c8 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:47:31.857244 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-10e03cd1-8e2e-48ce-9a73-0bad719b94c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:47:31.954312 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 324/644] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:47:31 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 11 11:47:31.956161 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9491ba56-d6b5-4d84-a8aa-ddeac578734a None None] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:31.956450 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9491ba56-d6b5-4d84-a8aa-ddeac578734a None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:31.956722 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9491ba56-d6b5-4d84-a8aa-ddeac578734a None None] PATH_INFO: `/` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:31.962363 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 325/645] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:47:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:47:31.966120 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2f9fe5-9d30-478c-b654-6a5793b74109 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:31.966120 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2f9fe5-9d30-478c-b654-6a5793b74109 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:31.966120 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2f9fe5-9d30-478c-b654-6a5793b74109 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:31.972103 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631297a5-4c71-4960-9fe5-c1ec2bc474e4 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:31.993645 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-fb2f9fe5-9d30-478c-b654-6a5793b74109 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:47:31.995387 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-fb2f9fe5-9d30-478c-b654-6a5793b74109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:47:32.015843 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220602f3-d9d1-449c-a723-00703c5a8128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:32.016517 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220602f3-d9d1-449c-a723-00703c5a8128 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:32.016856 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220602f3-d9d1-449c-a723-00703c5a8128 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:32.017249 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220602f3-d9d1-449c-a723-00703c5a8128 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:32.031413 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220602f3-d9d1-449c-a723-00703c5a8128 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:32.032686 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220602f3-d9d1-449c-a723-00703c5a8128 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:32.052268 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application [None req-220602f3-d9d1-449c-a723-00703c5a8128 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 11:47:32.052268 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:47:32.052268 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:47:32.052268 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:47:32.052268 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:47:32.052268 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:47:32.052268 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:47:32.052268 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:47:32.052268 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:47:32.052268 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:47:32.052268 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:47:32.052268 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:47:32.052268 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 11:47:32.052268 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 11:47:32.052268 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 11:47:32.052268 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 11:47:32.053294 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:47:32.053294 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:47:32.053294 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:47:32.053294 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:47:32.053294 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:47:32.053294 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:47:32.053294 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:47:32.053294 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application with Lock( Oct 11 11:47:32.053294 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:47:32.053294 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:47:32.053294 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:47:32.053294 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:47:32.053294 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:47:32.053294 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:47:32.053294 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:47:32.053294 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:47:32.053294 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 11:47:32.054200 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 11:47:32.054200 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 11:47:32.054200 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:47:32.054200 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 321/646] 10.222.0.59 () {56 vars in 1108 bytes} [Tue Oct 11 11:47:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 11 11:47:32.059375 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220602f3-d9d1-449c-a723-00703c5a8128 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:32.083384 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:32.098870 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 326/647] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:47:31 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 11:47:32.181463 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220602f3-d9d1-449c-a723-00703c5a8128 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:32.200221 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d9c857-5c7b-45bf-958b-dcc7d21090f2 None None] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:32.200705 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d9c857-5c7b-45bf-958b-dcc7d21090f2 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:32.201190 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d9c857-5c7b-45bf-958b-dcc7d21090f2 None None] PATH_INFO: `/` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:32.210479 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 327/648] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:47:32 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:47:32.213188 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9346174e-dac8-4445-bd7b-216957f593ed None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:32.213658 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9346174e-dac8-4445-bd7b-216957f593ed None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:32.214147 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9346174e-dac8-4445-bd7b-216957f593ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:32.220762 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91f8a33-eb4b-4fe8-a035-583cf47c5fc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:32.222262 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91f8a33-eb4b-4fe8-a035-583cf47c5fc5 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:32.222593 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91f8a33-eb4b-4fe8-a035-583cf47c5fc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:32.222930 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91f8a33-eb4b-4fe8-a035-583cf47c5fc5 None admin] PATH_INFO: `/v3/domains` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:32.223493 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91f8a33-eb4b-4fe8-a035-583cf47c5fc5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:32.223797 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91f8a33-eb4b-4fe8-a035-583cf47c5fc5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:32.225181 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91f8a33-eb4b-4fe8-a035-583cf47c5fc5 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:32.244786 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-9346174e-dac8-4445-bd7b-216957f593ed None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:47:32.245065 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 322/649] 10.222.0.59 () {56 vars in 1109 bytes} [Tue Oct 11 11:47:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 11:47:32.245065 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91f8a33-eb4b-4fe8-a035-583cf47c5fc5 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:32.245509 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-9346174e-dac8-4445-bd7b-216957f593ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:47:32.294498 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:32.329802 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 328/650] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:47:32 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 11 11:47:32.336483 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9346174e-dac8-4445-bd7b-216957f593ed None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:32.346266 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:32.374492 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91f8a33-eb4b-4fe8-a035-583cf47c5fc5 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:32.418286 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77395714-5d69-4c75-8cc4-3edbfd1fcfb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:32.418955 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77395714-5d69-4c75-8cc4-3edbfd1fcfb0 None admin] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:32.419232 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77395714-5d69-4c75-8cc4-3edbfd1fcfb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:32.419499 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77395714-5d69-4c75-8cc4-3edbfd1fcfb0 None admin] PATH_INFO: `/v3/services` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:32.420180 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77395714-5d69-4c75-8cc4-3edbfd1fcfb0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:32.421567 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77395714-5d69-4c75-8cc4-3edbfd1fcfb0 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:32.446635 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 323/651] 10.222.0.59 () {60 vars in 1141 bytes} [Tue Oct 11 11:47:32 2022] POST /identity/v3/services => generated 269 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 11:47:32.446635 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77395714-5d69-4c75-8cc4-3edbfd1fcfb0 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:32.497320 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9346174e-dac8-4445-bd7b-216957f593ed None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:32.564264 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fb50f9-5e31-49f4-bfbd-cf2b87917b24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:32.566103 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fb50f9-5e31-49f4-bfbd-cf2b87917b24 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:32.566705 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fb50f9-5e31-49f4-bfbd-cf2b87917b24 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:32.567266 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fb50f9-5e31-49f4-bfbd-cf2b87917b24 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:32.583537 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fb50f9-5e31-49f4-bfbd-cf2b87917b24 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:32.588575 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fb50f9-5e31-49f4-bfbd-cf2b87917b24 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:32.621690 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application [None req-f3fb50f9-5e31-49f4-bfbd-cf2b87917b24 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 11:47:32.621690 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:47:32.621690 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:47:32.621690 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:47:32.621690 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:47:32.621690 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:47:32.621690 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:47:32.621690 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:47:32.621690 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:47:32.621690 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:47:32.621690 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:47:32.621690 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:47:32.621690 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 11 11:47:32.621690 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 11 11:47:32.621690 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 11 11:47:32.623665 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 11 11:47:32.623665 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:47:32.623665 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:47:32.623665 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:47:32.623665 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:47:32.623665 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:47:32.623665 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:47:32.623665 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:47:32.623665 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application with Lock( Oct 11 11:47:32.623665 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:47:32.623665 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:47:32.623665 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:47:32.623665 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:47:32.623665 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:47:32.623665 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:47:32.623665 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:47:32.623665 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:47:32.625423 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 11 11:47:32.625423 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 11 11:47:32.625423 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 11 11:47:32.625423 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 11 11:47:32.625423 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 11 11:47:32.625423 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 11 11:47:32.625423 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 11:47:32.625423 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:47:32.625423 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 329/652] 10.222.0.59 () {56 vars in 1111 bytes} [Tue Oct 11 11:47:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:47:32.630704 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fb50f9-5e31-49f4-bfbd-cf2b87917b24 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:32.661262 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77395714-5d69-4c75-8cc4-3edbfd1fcfb0 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:32.706162 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:32.709283 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f96fb5-9cce-46e9-948f-1ff0f352d979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:32.709934 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f96fb5-9cce-46e9-948f-1ff0f352d979 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:32.710217 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f96fb5-9cce-46e9-948f-1ff0f352d979 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:32.710492 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f96fb5-9cce-46e9-948f-1ff0f352d979 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:32.712125 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f96fb5-9cce-46e9-948f-1ff0f352d979 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:32.712579 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f96fb5-9cce-46e9-948f-1ff0f352d979 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:32.722292 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application [None req-59f96fb5-9cce-46e9-948f-1ff0f352d979 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 11 11:47:32.722292 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:47:32.722292 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 11 11:47:32.722292 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 11 11:47:32.722292 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 11 11:47:32.722292 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 11 11:47:32.722292 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 11 11:47:32.722292 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 11:47:32.722292 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 11 11:47:32.722292 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:47:32.722292 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 11 11:47:32.722292 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:47:32.722292 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:47:32.722292 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:47:32.722292 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:47:32.722292 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:47:32.722292 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:47:32.724231 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:47:32.724231 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:47:32.724231 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:47:32.724231 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:47:32.724231 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:47:32.724231 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:47:32.724231 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 11 11:47:32.724231 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 11 11:47:32.724231 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 11 11:47:32.724231 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 11 11:47:32.724231 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:47:32.724231 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:47:32.724231 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:47:32.724231 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:47:32.724231 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:47:32.724231 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:47:32.725937 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:47:32.725937 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application with Lock( Oct 11 11:47:32.725937 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:47:32.725937 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:47:32.725937 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:47:32.725937 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:47:32.725937 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:47:32.725937 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:47:32.725937 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:47:32.725937 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:47:32.725937 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 11 11:47:32.725937 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 11:47:32.725937 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 11 11:47:32.725937 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:47:32.725937 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 324/653] 10.222.0.59 () {56 vars in 1114 bytes} [Tue Oct 11 11:47:32 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:47:32.729407 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f96fb5-9cce-46e9-948f-1ff0f352d979 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:32.733184 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:32.792358 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fb50f9-5e31-49f4-bfbd-cf2b87917b24 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:32.841775 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dd030c-2e93-4b66-be77-6215de8bf787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:32.842289 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dd030c-2e93-4b66-be77-6215de8bf787 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:32.842588 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dd030c-2e93-4b66-be77-6215de8bf787 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:32.842876 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dd030c-2e93-4b66-be77-6215de8bf787 None admin] PATH_INFO: `/v3/projects` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:32.843594 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dd030c-2e93-4b66-be77-6215de8bf787 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:32.843897 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dd030c-2e93-4b66-be77-6215de8bf787 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:32.845456 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dd030c-2e93-4b66-be77-6215de8bf787 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:32.859966 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 330/654] 10.222.0.59 () {56 vars in 1148 bytes} [Tue Oct 11 11:47:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:47:32.860658 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dd030c-2e93-4b66-be77-6215de8bf787 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:32.875905 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f96fb5-9cce-46e9-948f-1ff0f352d979 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:32.932944 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59e2174-e8b0-414f-8ef5-1ab629fbfe72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:32.934204 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59e2174-e8b0-414f-8ef5-1ab629fbfe72 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:32.934717 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59e2174-e8b0-414f-8ef5-1ab629fbfe72 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:32.935164 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59e2174-e8b0-414f-8ef5-1ab629fbfe72 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:32.935988 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59e2174-e8b0-414f-8ef5-1ab629fbfe72 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:32.938622 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59e2174-e8b0-414f-8ef5-1ab629fbfe72 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:32.950203 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application [None req-f59e2174-e8b0-414f-8ef5-1ab629fbfe72 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 11 11:47:32.950203 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:47:32.950203 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 11 11:47:32.950203 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 11 11:47:32.950203 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 11 11:47:32.950203 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 11 11:47:32.950203 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 11 11:47:32.950203 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 11:47:32.950203 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 11 11:47:32.950203 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:47:32.950203 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 11 11:47:32.950203 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:47:32.950203 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:47:32.950203 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:47:32.950203 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:47:32.950203 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:47:32.950203 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:47:32.951338 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:47:32.951338 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:47:32.951338 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:47:32.951338 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:47:32.951338 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:47:32.951338 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:47:32.951338 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 11 11:47:32.951338 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 11 11:47:32.951338 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 11 11:47:32.951338 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 11 11:47:32.951338 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:47:32.951338 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:47:32.951338 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:47:32.951338 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:47:32.951338 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:47:32.951338 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:47:32.952156 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:47:32.952156 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application with Lock( Oct 11 11:47:32.952156 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:47:32.952156 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:47:32.952156 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:47:32.952156 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:47:32.952156 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:47:32.952156 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:47:32.952156 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:47:32.952156 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:47:32.952156 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 11 11:47:32.952156 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 11:47:32.952156 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 11 11:47:32.952156 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:47:32.954940 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 325/655] 10.222.0.59 () {56 vars in 1126 bytes} [Tue Oct 11 11:47:32 2022] GET /identity/v3/services/object-store => generated 93 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:47:32.956196 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59e2174-e8b0-414f-8ef5-1ab629fbfe72 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:32.987783 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:32.999745 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dd030c-2e93-4b66-be77-6215de8bf787 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:33.032770 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:33.059489 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4a3d91-a48a-4440-845a-80d20dc9cb0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:33.060570 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4a3d91-a48a-4440-845a-80d20dc9cb0e None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:33.060931 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4a3d91-a48a-4440-845a-80d20dc9cb0e None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:33.061442 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4a3d91-a48a-4440-845a-80d20dc9cb0e None admin] PATH_INFO: `/v3/services` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:33.062582 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4a3d91-a48a-4440-845a-80d20dc9cb0e None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:33.063146 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4a3d91-a48a-4440-845a-80d20dc9cb0e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:33.065865 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4a3d91-a48a-4440-845a-80d20dc9cb0e None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:33.072857 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59e2174-e8b0-414f-8ef5-1ab629fbfe72 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:33.074502 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 331/656] 10.222.0.59 () {56 vars in 1114 bytes} [Tue Oct 11 11:47:32 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:47:33.076128 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4a3d91-a48a-4440-845a-80d20dc9cb0e None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:33.116494 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff54b96-bb18-49c8-aa40-4c7bf238445a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:33.117144 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff54b96-bb18-49c8-aa40-4c7bf238445a None admin] REQUEST_METHOD: `PUT` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:33.117351 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff54b96-bb18-49c8-aa40-4c7bf238445a None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:33.117620 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff54b96-bb18-49c8-aa40-4c7bf238445a None admin] PATH_INFO: `/v3/projects/26ecf552cc9a4e5ca1d380509683dc4a/users/8a19eceb62ce4157a4c5fe1866a89df9/roles/6a95127666ae4707bc09aca24e0d3d39` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:33.119973 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff54b96-bb18-49c8-aa40-4c7bf238445a None admin] RBAC: Authorizing `identity:create_grant(project_id=26ecf552cc9a4e5ca1d380509683dc4a, user_id=8a19eceb62ce4157a4c5fe1866a89df9, role_id=6a95127666ae4707bc09aca24e0d3d39)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:33.121527 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff54b96-bb18-49c8-aa40-4c7bf238445a None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:33.144672 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 326/657] 10.222.0.59 () {58 vars in 1439 bytes} [Tue Oct 11 11:47:32 2022] PUT /identity/v3/projects/26ecf552cc9a4e5ca1d380509683dc4a/users/8a19eceb62ce4157a4c5fe1866a89df9/roles/6a95127666ae4707bc09aca24e0d3d39 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 11:47:33.145730 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff54b96-bb18-49c8-aa40-4c7bf238445a None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:33.218633 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4a3d91-a48a-4440-845a-80d20dc9cb0e None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:33.281975 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f110d408-1442-484e-b665-4eaed4791638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:33.283158 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f110d408-1442-484e-b665-4eaed4791638 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:33.283648 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f110d408-1442-484e-b665-4eaed4791638 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:33.284147 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f110d408-1442-484e-b665-4eaed4791638 None admin] PATH_INFO: `/v3/services` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:33.285640 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f110d408-1442-484e-b665-4eaed4791638 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:33.286347 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f110d408-1442-484e-b665-4eaed4791638 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:33.289128 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f110d408-1442-484e-b665-4eaed4791638 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:33.299147 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 332/658] 10.222.0.59 () {56 vars in 1122 bytes} [Tue Oct 11 11:47:33 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:47:33.306121 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f110d408-1442-484e-b665-4eaed4791638 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:33.326173 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff54b96-bb18-49c8-aa40-4c7bf238445a None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:33.344061 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:33.369028 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:33.388679 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5235c604-9127-44f6-b287-5526ea6f4e9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:33.389744 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5235c604-9127-44f6-b287-5526ea6f4e9c None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:33.390236 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5235c604-9127-44f6-b287-5526ea6f4e9c None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:33.390778 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5235c604-9127-44f6-b287-5526ea6f4e9c None admin] PATH_INFO: `/v3/services` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:33.392276 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5235c604-9127-44f6-b287-5526ea6f4e9c None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:33.392830 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5235c604-9127-44f6-b287-5526ea6f4e9c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:33.395769 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5235c604-9127-44f6-b287-5526ea6f4e9c None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:33.405873 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 327/659] 10.222.0.59 () {56 vars in 1114 bytes} [Tue Oct 11 11:47:33 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:47:33.457764 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f110d408-1442-484e-b665-4eaed4791638 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:33.505507 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a081a3c8-6609-4f40-8e21-0718a4ce7554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:33.506583 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a081a3c8-6609-4f40-8e21-0718a4ce7554 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:33.507080 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a081a3c8-6609-4f40-8e21-0718a4ce7554 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:33.507565 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a081a3c8-6609-4f40-8e21-0718a4ce7554 None admin] PATH_INFO: `/v3/services` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:33.508725 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a081a3c8-6609-4f40-8e21-0718a4ce7554 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:33.509275 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a081a3c8-6609-4f40-8e21-0718a4ce7554 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:33.511512 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a081a3c8-6609-4f40-8e21-0718a4ce7554 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:33.520918 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 333/660] 10.222.0.59 () {56 vars in 1122 bytes} [Tue Oct 11 11:47:33 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 11:47:35.550074 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb34b56-1229-44db-a469-4dbc4acb0d2c None None] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:35.550672 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb34b56-1229-44db-a469-4dbc4acb0d2c None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:35.550672 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb34b56-1229-44db-a469-4dbc4acb0d2c None None] PATH_INFO: `/` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:35.552402 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 328/661] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:47:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:47:35.558512 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a36421-7a13-4262-885e-79b1add40b58 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:35.559014 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a36421-7a13-4262-885e-79b1add40b58 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:35.559117 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a36421-7a13-4262-885e-79b1add40b58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:35.580061 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-10a36421-7a13-4262-885e-79b1add40b58 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:47:35.581344 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-10a36421-7a13-4262-885e-79b1add40b58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:47:35.611678 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e1ebb0-b75d-4e5d-be64-1977745fa1c0 None None] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:35.612160 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e1ebb0-b75d-4e5d-be64-1977745fa1c0 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:35.612638 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e1ebb0-b75d-4e5d-be64-1977745fa1c0 None None] PATH_INFO: `/` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:35.615405 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 329/662] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:47:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:47:35.623478 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda504c5-9635-4544-a20d-01409c6f21b7 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:35.623962 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda504c5-9635-4544-a20d-01409c6f21b7 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:35.624396 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda504c5-9635-4544-a20d-01409c6f21b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:35.654275 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-fda504c5-9635-4544-a20d-01409c6f21b7 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:47:35.656480 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-fda504c5-9635-4544-a20d-01409c6f21b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:47:35.662694 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 334/663] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:47:35 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 11:47:35.765121 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 330/664] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:47:35 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 11:47:35.796840 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d025a6b-f180-4b5a-96c7-c260c32f5984 None None] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:35.797145 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d025a6b-f180-4b5a-96c7-c260c32f5984 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:35.797396 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d025a6b-f180-4b5a-96c7-c260c32f5984 None None] PATH_INFO: `/` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:35.799156 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 335/665] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:47:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:47:35.804954 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b859ec1-dfea-4a03-95c5-1e73f91a103e None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:35.805201 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b859ec1-dfea-4a03-95c5-1e73f91a103e None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:35.805449 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b859ec1-dfea-4a03-95c5-1e73f91a103e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:35.824797 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-4b859ec1-dfea-4a03-95c5-1e73f91a103e None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:47:35.826004 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-4b859ec1-dfea-4a03-95c5-1e73f91a103e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:47:35.847753 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f6cf11-b382-4acd-bd49-8a30abab331d None None] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:35.848243 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f6cf11-b382-4acd-bd49-8a30abab331d None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:35.848715 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f6cf11-b382-4acd-bd49-8a30abab331d None None] PATH_INFO: `/` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:35.851525 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 336/666] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:47:35 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:47:35.860483 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b93081-7fa9-4258-82d7-1a5f95b9b6c9 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:35.861105 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b93081-7fa9-4258-82d7-1a5f95b9b6c9 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:35.861577 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b93081-7fa9-4258-82d7-1a5f95b9b6c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:35.902179 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 331/667] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:47:35 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 11:47:35.902658 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-a2b93081-7fa9-4258-82d7-1a5f95b9b6c9 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:47:35.904811 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-a2b93081-7fa9-4258-82d7-1a5f95b9b6c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:47:35.932195 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:35.945301 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b859ec1-dfea-4a03-95c5-1e73f91a103e None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:36.007178 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 337/668] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:47:35 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 11:47:36.015702 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:36.073796 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b859ec1-dfea-4a03-95c5-1e73f91a103e None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:36.080670 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b93081-7fa9-4258-82d7-1a5f95b9b6c9 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:36.113693 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0008529-a409-4bc9-84be-3175899a5f8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:36.114353 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0008529-a409-4bc9-84be-3175899a5f8d None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:36.114664 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0008529-a409-4bc9-84be-3175899a5f8d None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:36.115104 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0008529-a409-4bc9-84be-3175899a5f8d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:36.127411 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0008529-a409-4bc9-84be-3175899a5f8d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 11:47:36.132625 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application [None req-f0008529-a409-4bc9-84be-3175899a5f8d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 11 11:47:36.132625 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:47:36.132625 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:47:36.132625 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:47:36.132625 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:47:36.132625 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:47:36.132625 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:47:36.132625 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:47:36.132625 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:47:36.132625 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:47:36.132625 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:47:36.132625 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:47:36.132625 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 11:47:36.132625 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 11:47:36.132625 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 11:47:36.132625 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 11:47:36.133687 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 11:47:36.133687 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 11:47:36.133687 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 11:47:36.133687 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 11:47:36.133687 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:47:36.133687 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:47:36.133687 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:47:36.133687 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:47:36.133687 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:47:36.133687 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:47:36.133687 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:47:36.133687 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application with Lock( Oct 11 11:47:36.133687 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:47:36.133687 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:47:36.133687 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:47:36.133687 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:47:36.139901 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:47:36.139901 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:47:36.139901 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:47:36.139901 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:47:36.139901 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 11:47:36.139901 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 11:47:36.139901 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 11:47:36.139901 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 11:47:36.139901 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 11:47:36.139901 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 11:47:36.139901 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 11 11:47:36.139901 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:47:36.139901 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 332/669] 10.222.0.59 () {56 vars in 1120 bytes} [Tue Oct 11 11:47:35 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 11 11:47:36.140874 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0008529-a409-4bc9-84be-3175899a5f8d None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:36.201801 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b93081-7fa9-4258-82d7-1a5f95b9b6c9 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:36.244532 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc9d8d9-a45d-4cc4-8789-107f2a6436c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:36.245183 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc9d8d9-a45d-4cc4-8789-107f2a6436c3 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:36.245467 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc9d8d9-a45d-4cc4-8789-107f2a6436c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:36.245740 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc9d8d9-a45d-4cc4-8789-107f2a6436c3 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:36.246330 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc9d8d9-a45d-4cc4-8789-107f2a6436c3 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:36.247780 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc9d8d9-a45d-4cc4-8789-107f2a6436c3 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:36.257278 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application [None req-8fc9d8d9-a45d-4cc4-8789-107f2a6436c3 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 11 11:47:36.257278 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:47:36.257278 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 11 11:47:36.257278 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 11 11:47:36.257278 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 11 11:47:36.257278 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 11 11:47:36.257278 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 11 11:47:36.257278 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 11:47:36.257278 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 11 11:47:36.257278 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:47:36.257278 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 11 11:47:36.257278 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:47:36.257278 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:47:36.257278 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:47:36.257278 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:47:36.257278 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:47:36.258278 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:47:36.258278 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:47:36.258278 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:47:36.258278 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:47:36.258278 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:47:36.258278 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:47:36.258278 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:47:36.258278 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 11 11:47:36.258278 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 11 11:47:36.258278 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 11 11:47:36.258278 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 11 11:47:36.258278 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:47:36.258278 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:47:36.258278 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:47:36.258278 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:47:36.258278 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:47:36.258278 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:47:36.259221 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:47:36.259221 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application with Lock( Oct 11 11:47:36.259221 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:47:36.259221 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:47:36.259221 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:47:36.259221 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:47:36.259221 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:47:36.259221 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:47:36.259221 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:47:36.259221 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:47:36.259221 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 11 11:47:36.259221 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 11:47:36.259221 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 11 11:47:36.259221 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:47:36.259221 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 338/670] 10.222.0.59 () {56 vars in 1132 bytes} [Tue Oct 11 11:47:36 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 11 11:47:36.260005 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc9d8d9-a45d-4cc4-8789-107f2a6436c3 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:36.277687 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:36.280924 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0008529-a409-4bc9-84be-3175899a5f8d None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:36.294231 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:36.325209 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63bf4fc-f718-4d2a-94ae-20269bac8708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:36.325804 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63bf4fc-f718-4d2a-94ae-20269bac8708 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:36.326079 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63bf4fc-f718-4d2a-94ae-20269bac8708 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:36.326362 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63bf4fc-f718-4d2a-94ae-20269bac8708 None admin] PATH_INFO: `/v3/roles` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:36.327106 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63bf4fc-f718-4d2a-94ae-20269bac8708 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:36.327419 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63bf4fc-f718-4d2a-94ae-20269bac8708 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:36.328823 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63bf4fc-f718-4d2a-94ae-20269bac8708 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:36.346157 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 333/671] 10.222.0.59 () {56 vars in 1115 bytes} [Tue Oct 11 11:47:36 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 11:47:36.346157 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63bf4fc-f718-4d2a-94ae-20269bac8708 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:36.404133 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc9d8d9-a45d-4cc4-8789-107f2a6436c3 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:36.467505 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89ebbc3-1497-43dc-9555-23d10e03a0c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:36.468539 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89ebbc3-1497-43dc-9555-23d10e03a0c8 None admin] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:36.469013 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89ebbc3-1497-43dc-9555-23d10e03a0c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:36.469489 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89ebbc3-1497-43dc-9555-23d10e03a0c8 None admin] PATH_INFO: `/v3/services` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:36.470782 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89ebbc3-1497-43dc-9555-23d10e03a0c8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:36.473877 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89ebbc3-1497-43dc-9555-23d10e03a0c8 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:36.476058 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63bf4fc-f718-4d2a-94ae-20269bac8708 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:36.499183 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 339/672] 10.222.0.59 () {60 vars in 1141 bytes} [Tue Oct 11 11:47:36 2022] POST /identity/v3/services => generated 251 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 11:47:36.500647 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89ebbc3-1497-43dc-9555-23d10e03a0c8 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:36.549878 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ab232a-c547-4785-9bb5-f9faa9452695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:36.550659 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ab232a-c547-4785-9bb5-f9faa9452695 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:36.551545 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ab232a-c547-4785-9bb5-f9faa9452695 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:36.551545 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ab232a-c547-4785-9bb5-f9faa9452695 None admin] PATH_INFO: `/v3/services` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:36.552000 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ab232a-c547-4785-9bb5-f9faa9452695 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:36.552368 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ab232a-c547-4785-9bb5-f9faa9452695 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:36.554171 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ab232a-c547-4785-9bb5-f9faa9452695 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:36.561913 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 334/673] 10.222.0.59 () {56 vars in 1126 bytes} [Tue Oct 11 11:47:36 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:47:36.563018 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ab232a-c547-4785-9bb5-f9faa9452695 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:36.622733 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:36.630166 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:36.672270 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89ebbc3-1497-43dc-9555-23d10e03a0c8 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:36.714319 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ff5de6-5baf-40c3-b9f2-4e95491389fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:36.715132 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ff5de6-5baf-40c3-b9f2-4e95491389fa None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:36.715412 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ff5de6-5baf-40c3-b9f2-4e95491389fa None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:36.715672 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ff5de6-5baf-40c3-b9f2-4e95491389fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:36.734295 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ff5de6-5baf-40c3-b9f2-4e95491389fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:36.736924 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ff5de6-5baf-40c3-b9f2-4e95491389fa None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:36.754396 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application [None req-25ff5de6-5baf-40c3-b9f2-4e95491389fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 11:47:36.754396 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:47:36.754396 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:47:36.754396 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:47:36.754396 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:47:36.754396 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:47:36.754396 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:47:36.754396 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:47:36.754396 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:47:36.754396 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:47:36.754396 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:47:36.754396 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:47:36.754396 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 11:47:36.754396 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 11:47:36.754396 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 11:47:36.754396 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 11:47:36.755427 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:47:36.755427 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:47:36.755427 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:47:36.755427 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:47:36.755427 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:47:36.755427 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:47:36.755427 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:47:36.755427 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application with Lock( Oct 11 11:47:36.755427 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:47:36.755427 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:47:36.755427 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:47:36.755427 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:47:36.755427 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:47:36.755427 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:47:36.755427 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:47:36.755427 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:47:36.755427 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 11:47:36.756571 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 11:47:36.756571 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 11:47:36.756571 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:47:36.756571 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 340/674] 10.222.0.59 () {56 vars in 1108 bytes} [Tue Oct 11 11:47:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:47:36.757212 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ff5de6-5baf-40c3-b9f2-4e95491389fa None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:36.772813 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ab232a-c547-4785-9bb5-f9faa9452695 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:36.813936 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd68fd70-6a00-4a34-aacd-cd937d71ee97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:36.814565 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd68fd70-6a00-4a34-aacd-cd937d71ee97 None admin] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:36.814868 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd68fd70-6a00-4a34-aacd-cd937d71ee97 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:36.815135 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd68fd70-6a00-4a34-aacd-cd937d71ee97 None admin] PATH_INFO: `/v3/services` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:36.815822 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd68fd70-6a00-4a34-aacd-cd937d71ee97 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:36.817180 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd68fd70-6a00-4a34-aacd-cd937d71ee97 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:36.839574 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 335/675] 10.222.0.59 () {60 vars in 1141 bytes} [Tue Oct 11 11:47:36 2022] POST /identity/v3/services => generated 241 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 11:47:36.841233 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd68fd70-6a00-4a34-aacd-cd937d71ee97 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:36.928705 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ff5de6-5baf-40c3-b9f2-4e95491389fa None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:36.972284 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:37.005735 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f33997-d838-4dbd-804b-69ebbc3a5b6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:37.006137 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f33997-d838-4dbd-804b-69ebbc3a5b6c None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:37.006625 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f33997-d838-4dbd-804b-69ebbc3a5b6c None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:37.007130 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f33997-d838-4dbd-804b-69ebbc3a5b6c None admin] PATH_INFO: `/v3/services` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:37.008297 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f33997-d838-4dbd-804b-69ebbc3a5b6c None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:37.008815 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f33997-d838-4dbd-804b-69ebbc3a5b6c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:37.011359 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f33997-d838-4dbd-804b-69ebbc3a5b6c None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:37.021203 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 341/676] 10.222.0.59 () {56 vars in 1126 bytes} [Tue Oct 11 11:47:36 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:47:37.030602 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f33997-d838-4dbd-804b-69ebbc3a5b6c None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:37.055959 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:37.070405 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd68fd70-6a00-4a34-aacd-cd937d71ee97 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:37.140511 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d273be46-bd83-4bf0-be84-7a76684afad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:37.141698 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d273be46-bd83-4bf0-be84-7a76684afad8 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:37.142153 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d273be46-bd83-4bf0-be84-7a76684afad8 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:37.142687 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d273be46-bd83-4bf0-be84-7a76684afad8 None admin] PATH_INFO: `/v3/domains` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:37.143972 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d273be46-bd83-4bf0-be84-7a76684afad8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:37.144485 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d273be46-bd83-4bf0-be84-7a76684afad8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:37.147173 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d273be46-bd83-4bf0-be84-7a76684afad8 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:37.173549 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 336/677] 10.222.0.59 () {56 vars in 1109 bytes} [Tue Oct 11 11:47:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:47:37.182920 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d273be46-bd83-4bf0-be84-7a76684afad8 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:37.231451 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f33997-d838-4dbd-804b-69ebbc3a5b6c None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:37.276638 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe35fba9-d236-462d-b9b9-594c487ec05b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:37.277264 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe35fba9-d236-462d-b9b9-594c487ec05b None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:37.277544 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe35fba9-d236-462d-b9b9-594c487ec05b None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:37.277810 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe35fba9-d236-462d-b9b9-594c487ec05b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:37.278532 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe35fba9-d236-462d-b9b9-594c487ec05b None admin] RBAC: Adding query filter params (interface=public, service_id=207b7a07e0f64a3a8fb0f8155daf9a9c, region_id=RegionOne) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:37.278875 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe35fba9-d236-462d-b9b9-594c487ec05b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:37.280855 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe35fba9-d236-462d-b9b9-594c487ec05b None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:37.291711 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 342/678] 10.222.0.59 () {56 vars in 1251 bytes} [Tue Oct 11 11:47:37 2022] GET /identity/v3/endpoints?service_id=207b7a07e0f64a3a8fb0f8155daf9a9c&interface=public®ion_id=RegionOne => generated 194 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:47:37.302456 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe35fba9-d236-462d-b9b9-594c487ec05b None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:37.353506 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d273be46-bd83-4bf0-be84-7a76684afad8 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:37.391397 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:37.402349 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0d135b-3b96-4f71-88d8-41a7185e5b17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:37.403047 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0d135b-3b96-4f71-88d8-41a7185e5b17 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:37.403332 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0d135b-3b96-4f71-88d8-41a7185e5b17 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:37.403605 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0d135b-3b96-4f71-88d8-41a7185e5b17 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:37.413934 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0d135b-3b96-4f71-88d8-41a7185e5b17 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:37.416092 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0d135b-3b96-4f71-88d8-41a7185e5b17 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:37.432638 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application [None req-4e0d135b-3b96-4f71-88d8-41a7185e5b17 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 11 11:47:37.432638 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:47:37.432638 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:47:37.432638 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:47:37.432638 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:47:37.432638 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:47:37.432638 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:47:37.432638 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:47:37.432638 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:47:37.432638 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:47:37.432638 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:47:37.432638 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:47:37.432638 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 11 11:47:37.432638 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 11 11:47:37.432638 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 11 11:47:37.432638 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 11 11:47:37.434348 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:47:37.434348 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:47:37.434348 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 11 11:47:37.434348 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 11:47:37.434348 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 11 11:47:37.434348 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 11:47:37.434348 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:47:37.434348 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:47:37.434348 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:47:37.434348 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:47:37.434348 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:47:37.434348 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application with Lock( Oct 11 11:47:37.434348 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:47:37.434348 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:47:37.434348 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:47:37.434348 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:47:37.434348 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:47:37.434348 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:47:37.436183 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:47:37.436183 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:47:37.436183 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 11 11:47:37.436183 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 11 11:47:37.436183 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 11 11:47:37.436183 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 11 11:47:37.436183 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 11 11:47:37.436183 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 11 11:47:37.436183 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 11 11:47:37.436183 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:47:37.439086 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 337/679] 10.222.0.59 () {56 vars in 1117 bytes} [Tue Oct 11 11:47:37 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 11 11:47:37.440623 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0d135b-3b96-4f71-88d8-41a7185e5b17 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:37.462681 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:37.513788 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe35fba9-d236-462d-b9b9-594c487ec05b None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:37.572822 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d734099-8847-40ca-a842-a32ed67ce2b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:37.574236 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d734099-8847-40ca-a842-a32ed67ce2b3 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:37.575031 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d734099-8847-40ca-a842-a32ed67ce2b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:37.575819 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d734099-8847-40ca-a842-a32ed67ce2b3 None admin] PATH_INFO: `/v3/services` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:37.577168 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d734099-8847-40ca-a842-a32ed67ce2b3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:37.580030 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d734099-8847-40ca-a842-a32ed67ce2b3 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:37.584270 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0d135b-3b96-4f71-88d8-41a7185e5b17 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:37.595230 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 343/680] 10.222.0.59 () {56 vars in 1087 bytes} [Tue Oct 11 11:47:37 2022] GET /identity/v3/services => generated 1497 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 11:47:37.626602 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ea349a-8a64-456f-8f8f-aa199d737565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:37.627284 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ea349a-8a64-456f-8f8f-aa199d737565 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:37.627567 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ea349a-8a64-456f-8f8f-aa199d737565 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:37.627843 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ea349a-8a64-456f-8f8f-aa199d737565 None admin] PATH_INFO: `/v3/users` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:37.628615 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ea349a-8a64-456f-8f8f-aa199d737565 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:37.628920 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ea349a-8a64-456f-8f8f-aa199d737565 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:37.630518 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ea349a-8a64-456f-8f8f-aa199d737565 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:37.657785 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 338/681] 10.222.0.59 () {56 vars in 1149 bytes} [Tue Oct 11 11:47:37 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:47:37.659707 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d734099-8847-40ca-a842-a32ed67ce2b3 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:37.839109 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d734099-8847-40ca-a842-a32ed67ce2b3 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:37.861646 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:37.906157 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543bece7-1fcf-4749-bf56-21f8056a0747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:37.907259 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543bece7-1fcf-4749-bf56-21f8056a0747 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:37.907729 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543bece7-1fcf-4749-bf56-21f8056a0747 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:37.908181 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543bece7-1fcf-4749-bf56-21f8056a0747 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:38.055496 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543bece7-1fcf-4749-bf56-21f8056a0747 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:38.058423 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543bece7-1fcf-4749-bf56-21f8056a0747 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:38.218815 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 344/682] 10.222.0.59 () {58 vars in 1282 bytes} [Tue Oct 11 11:47:37 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 560 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 11 11:47:38.225106 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ea349a-8a64-456f-8f8f-aa199d737565 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:38.310663 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:38.349193 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ea349a-8a64-456f-8f8f-aa199d737565 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:38.391112 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cc6ea7-5f3f-4077-a074-eb95386269c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:38.392320 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cc6ea7-5f3f-4077-a074-eb95386269c8 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:38.392825 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cc6ea7-5f3f-4077-a074-eb95386269c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:38.393297 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cc6ea7-5f3f-4077-a074-eb95386269c8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:38.405678 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cc6ea7-5f3f-4077-a074-eb95386269c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:38.407604 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cc6ea7-5f3f-4077-a074-eb95386269c8 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:38.427820 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application [None req-d8cc6ea7-5f3f-4077-a074-eb95386269c8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 11:47:38.427820 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:47:38.427820 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:47:38.427820 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:47:38.427820 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:47:38.427820 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:47:38.427820 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:47:38.427820 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:47:38.427820 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:47:38.427820 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:47:38.427820 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:47:38.427820 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:47:38.427820 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 11:47:38.427820 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 11:47:38.427820 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 11:47:38.427820 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 11:47:38.428842 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:47:38.428842 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:47:38.428842 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:47:38.428842 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:47:38.428842 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:47:38.428842 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:47:38.428842 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:47:38.428842 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application with Lock( Oct 11 11:47:38.428842 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:47:38.428842 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:47:38.428842 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:47:38.428842 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:47:38.428842 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:47:38.428842 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:47:38.428842 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:47:38.428842 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:47:38.428842 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 11:47:38.429720 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 11:47:38.429720 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 11:47:38.429720 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:47:38.438446 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 339/683] 10.222.0.59 () {56 vars in 1108 bytes} [Tue Oct 11 11:47:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:47:38.440944 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543bece7-1fcf-4749-bf56-21f8056a0747 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:38.463342 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:38.630979 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543bece7-1fcf-4749-bf56-21f8056a0747 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:38.699266 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c45c1dd-13db-4bec-9ba9-a535d5f1e926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:38.700376 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c45c1dd-13db-4bec-9ba9-a535d5f1e926 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:38.700852 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c45c1dd-13db-4bec-9ba9-a535d5f1e926 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:38.701336 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c45c1dd-13db-4bec-9ba9-a535d5f1e926 None admin] PATH_INFO: `/v3/domains` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:38.702528 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c45c1dd-13db-4bec-9ba9-a535d5f1e926 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:38.703120 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c45c1dd-13db-4bec-9ba9-a535d5f1e926 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:38.705794 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c45c1dd-13db-4bec-9ba9-a535d5f1e926 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:38.723033 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 345/684] 10.222.0.59 () {56 vars in 1109 bytes} [Tue Oct 11 11:47:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:47:38.731875 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cc6ea7-5f3f-4077-a074-eb95386269c8 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:38.846419 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cc6ea7-5f3f-4077-a074-eb95386269c8 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:38.873666 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:38.889528 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa54685-b9be-4fbd-9d3e-a3f5022a342f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:38.890741 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa54685-b9be-4fbd-9d3e-a3f5022a342f None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:38.891252 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa54685-b9be-4fbd-9d3e-a3f5022a342f None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:38.891770 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa54685-b9be-4fbd-9d3e-a3f5022a342f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:38.900563 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa54685-b9be-4fbd-9d3e-a3f5022a342f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:38.902569 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa54685-b9be-4fbd-9d3e-a3f5022a342f None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:38.916297 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application [None req-bfa54685-b9be-4fbd-9d3e-a3f5022a342f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 11:47:38.916297 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:47:38.916297 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:47:38.916297 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:47:38.916297 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:47:38.916297 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:47:38.916297 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:47:38.916297 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:47:38.916297 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:47:38.916297 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:47:38.916297 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:47:38.916297 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:47:38.916297 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 11 11:47:38.916297 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 11 11:47:38.916297 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 11 11:47:38.917242 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 11 11:47:38.917242 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:47:38.917242 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:47:38.917242 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:47:38.917242 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:47:38.917242 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:47:38.917242 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:47:38.917242 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:47:38.917242 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application with Lock( Oct 11 11:47:38.917242 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:47:38.917242 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:47:38.917242 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:47:38.917242 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:47:38.917242 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:47:38.917242 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:47:38.917242 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:47:38.917242 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:47:38.918266 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 11 11:47:38.918266 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 11 11:47:38.918266 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 11 11:47:38.918266 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 11 11:47:38.918266 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 11 11:47:38.918266 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 11 11:47:38.918266 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 11:47:38.918266 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:47:38.925405 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 340/685] 10.222.0.59 () {56 vars in 1111 bytes} [Tue Oct 11 11:47:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:47:38.928399 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c45c1dd-13db-4bec-9ba9-a535d5f1e926 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:38.999892 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69899891-5e8e-4e71-a877-ce9c624d369a None None] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:39.000229 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69899891-5e8e-4e71-a877-ce9c624d369a None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:39.000460 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69899891-5e8e-4e71-a877-ce9c624d369a None None] PATH_INFO: `/` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:39.002170 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 341/686] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:47:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:47:39.008586 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5ad85b-ff6e-4f3e-ace4-ecb26e1b18a7 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:39.009261 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5ad85b-ff6e-4f3e-ace4-ecb26e1b18a7 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:39.009737 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5ad85b-ff6e-4f3e-ace4-ecb26e1b18a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:39.014710 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:39.042081 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-4d5ad85b-ff6e-4f3e-ace4-ecb26e1b18a7 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:47:39.043927 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-4d5ad85b-ff6e-4f3e-ace4-ecb26e1b18a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:47:39.052153 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c45c1dd-13db-4bec-9ba9-a535d5f1e926 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:39.092600 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d0f30a-afab-4e3a-b553-f394d26ad13a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:39.093545 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d0f30a-afab-4e3a-b553-f394d26ad13a None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:39.093983 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d0f30a-afab-4e3a-b553-f394d26ad13a None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:39.094403 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d0f30a-afab-4e3a-b553-f394d26ad13a None admin] PATH_INFO: `/v3/projects` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:39.095390 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d0f30a-afab-4e3a-b553-f394d26ad13a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:39.095842 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d0f30a-afab-4e3a-b553-f394d26ad13a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:39.097544 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d0f30a-afab-4e3a-b553-f394d26ad13a None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:39.120434 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 346/687] 10.222.0.59 () {56 vars in 1148 bytes} [Tue Oct 11 11:47:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:47:39.122067 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d0f30a-afab-4e3a-b553-f394d26ad13a None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:39.142759 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 342/688] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:47:39 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 11 11:47:39.258873 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d0f30a-afab-4e3a-b553-f394d26ad13a None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:39.285165 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75fc4dc-724b-4a17-bcd9-689c22caaadb None None] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:39.285633 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75fc4dc-724b-4a17-bcd9-689c22caaadb None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:39.286167 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75fc4dc-724b-4a17-bcd9-689c22caaadb None None] PATH_INFO: `/` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:39.289164 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 343/689] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:47:39 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:47:39.298587 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931cf00e-da8b-406e-81d7-d495c66ff279 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:39.299089 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931cf00e-da8b-406e-81d7-d495c66ff279 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:39.299573 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931cf00e-da8b-406e-81d7-d495c66ff279 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:39.311162 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:39.319369 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b229db16-730a-48c8-aecb-134a4256c996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:39.320945 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b229db16-730a-48c8-aecb-134a4256c996 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:39.321473 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b229db16-730a-48c8-aecb-134a4256c996 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:39.321977 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b229db16-730a-48c8-aecb-134a4256c996 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:39.322976 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b229db16-730a-48c8-aecb-134a4256c996 None admin] RBAC: Adding query filter params (role.id=6a95127666ae4707bc09aca24e0d3d39, scope.project.id=26ecf552cc9a4e5ca1d380509683dc4a, user.id=8a19eceb62ce4157a4c5fe1866a89df9) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:39.323494 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b229db16-730a-48c8-aecb-134a4256c996 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:39.325479 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b229db16-730a-48c8-aecb-134a4256c996 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:39.329631 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-931cf00e-da8b-406e-81d7-d495c66ff279 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:47:39.331848 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-931cf00e-da8b-406e-81d7-d495c66ff279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:47:39.338876 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 347/690] 10.222.0.59 () {56 vars in 1374 bytes} [Tue Oct 11 11:47:39 2022] GET /identity/v3/role_assignments?user.id=8a19eceb62ce4157a4c5fe1866a89df9&scope.project.id=26ecf552cc9a4e5ca1d380509683dc4a&role.id=6a95127666ae4707bc09aca24e0d3d39 => generated 608 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:47:39.419431 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b229db16-730a-48c8-aecb-134a4256c996 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:39.465395 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 344/691] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:47:39 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 11:47:39.547054 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b229db16-730a-48c8-aecb-134a4256c996 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:39.586442 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd76b2d-7243-4abb-aa0a-b715bbea5302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:39.587139 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd76b2d-7243-4abb-aa0a-b715bbea5302 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:39.587419 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd76b2d-7243-4abb-aa0a-b715bbea5302 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:39.587701 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd76b2d-7243-4abb-aa0a-b715bbea5302 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:39.588297 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd76b2d-7243-4abb-aa0a-b715bbea5302 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:39.589717 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd76b2d-7243-4abb-aa0a-b715bbea5302 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:39.599786 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application [None req-7cd76b2d-7243-4abb-aa0a-b715bbea5302 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 11 11:47:39.599786 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:47:39.599786 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 11 11:47:39.599786 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 11 11:47:39.599786 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 11 11:47:39.599786 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 11 11:47:39.599786 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 11 11:47:39.599786 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 11:47:39.599786 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 11 11:47:39.599786 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:47:39.599786 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 11 11:47:39.599786 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:47:39.599786 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:47:39.599786 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:47:39.599786 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:47:39.599786 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:47:39.599786 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:47:39.600908 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:47:39.600908 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:47:39.600908 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:47:39.600908 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:47:39.600908 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:47:39.600908 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:47:39.600908 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 11 11:47:39.600908 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 11 11:47:39.600908 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 11 11:47:39.600908 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 11 11:47:39.600908 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:47:39.600908 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:47:39.600908 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:47:39.600908 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:47:39.600908 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:47:39.600908 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:47:39.601796 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:47:39.601796 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application with Lock( Oct 11 11:47:39.601796 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:47:39.601796 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:47:39.601796 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:47:39.601796 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:47:39.601796 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:47:39.601796 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:47:39.601796 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:47:39.601796 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:47:39.601796 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 11 11:47:39.601796 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 11:47:39.601796 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 11 11:47:39.601796 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:47:39.601796 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 348/692] 10.222.0.59 () {56 vars in 1114 bytes} [Tue Oct 11 11:47:39 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 11 11:47:39.608971 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931cf00e-da8b-406e-81d7-d495c66ff279 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:39.659756 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:39.721645 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931cf00e-da8b-406e-81d7-d495c66ff279 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:39.758721 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6027ed-999f-4378-838d-e2d52c62c2d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:39.760171 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6027ed-999f-4378-838d-e2d52c62c2d4 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:39.760677 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6027ed-999f-4378-838d-e2d52c62c2d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:39.761212 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6027ed-999f-4378-838d-e2d52c62c2d4 None admin] PATH_INFO: `/v3/services` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:39.762048 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6027ed-999f-4378-838d-e2d52c62c2d4 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:39.762597 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6027ed-999f-4378-838d-e2d52c62c2d4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:39.764181 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6027ed-999f-4378-838d-e2d52c62c2d4 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:39.770084 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 345/693] 10.222.0.59 () {56 vars in 1114 bytes} [Tue Oct 11 11:47:39 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:47:39.775791 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd76b2d-7243-4abb-aa0a-b715bbea5302 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:39.840411 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:39.886009 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd76b2d-7243-4abb-aa0a-b715bbea5302 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:39.928574 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51ef39e-2092-4fad-8c2a-b4390de3da65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:39.929589 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51ef39e-2092-4fad-8c2a-b4390de3da65 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:39.930063 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51ef39e-2092-4fad-8c2a-b4390de3da65 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:39.930512 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51ef39e-2092-4fad-8c2a-b4390de3da65 None admin] PATH_INFO: `/v3/services` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:39.931399 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51ef39e-2092-4fad-8c2a-b4390de3da65 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:39.931934 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51ef39e-2092-4fad-8c2a-b4390de3da65 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:39.933632 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51ef39e-2092-4fad-8c2a-b4390de3da65 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:39.940007 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 349/694] 10.222.0.59 () {56 vars in 1114 bytes} [Tue Oct 11 11:47:39 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:47:39.949998 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6027ed-999f-4378-838d-e2d52c62c2d4 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:39.967710 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51ef39e-2092-4fad-8c2a-b4390de3da65 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:40.104024 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6027ed-999f-4378-838d-e2d52c62c2d4 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:40.120773 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:40.125592 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51ef39e-2092-4fad-8c2a-b4390de3da65 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:40.149950 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:40.163159 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f16a2d-f237-4df8-ad6e-83c49af29d67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:40.163809 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f16a2d-f237-4df8-ad6e-83c49af29d67 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:40.164097 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f16a2d-f237-4df8-ad6e-83c49af29d67 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:40.164346 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f16a2d-f237-4df8-ad6e-83c49af29d67 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:40.164936 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f16a2d-f237-4df8-ad6e-83c49af29d67 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:40.166481 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f16a2d-f237-4df8-ad6e-83c49af29d67 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:40.175880 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application [None req-a5f16a2d-f237-4df8-ad6e-83c49af29d67 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 11 11:47:40.175880 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:47:40.175880 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 11 11:47:40.175880 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 11 11:47:40.175880 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 11 11:47:40.175880 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 11 11:47:40.175880 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 11 11:47:40.175880 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 11:47:40.175880 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 11 11:47:40.175880 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:47:40.175880 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 11 11:47:40.175880 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:47:40.175880 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:47:40.175880 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:47:40.175880 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:47:40.175880 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:47:40.175880 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:47:40.177847 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:47:40.177847 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:47:40.177847 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:47:40.177847 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:47:40.177847 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:47:40.177847 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:47:40.177847 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 11 11:47:40.177847 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 11 11:47:40.177847 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 11 11:47:40.177847 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 11 11:47:40.177847 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:47:40.177847 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:47:40.177847 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:47:40.177847 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:47:40.177847 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:47:40.177847 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:47:40.180568 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:47:40.180568 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application with Lock( Oct 11 11:47:40.180568 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:47:40.180568 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:47:40.180568 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:47:40.180568 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:47:40.180568 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:47:40.180568 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:47:40.180568 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:47:40.180568 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:47:40.180568 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 11 11:47:40.180568 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 11:47:40.180568 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 11 11:47:40.180568 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:47:40.180568 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 350/695] 10.222.0.59 () {56 vars in 1126 bytes} [Tue Oct 11 11:47:39 2022] GET /identity/v3/services/object-store => generated 93 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:47:40.183862 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0194891c-e31d-430a-a3bb-758816656213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:40.183862 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0194891c-e31d-430a-a3bb-758816656213 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:40.183862 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0194891c-e31d-430a-a3bb-758816656213 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:40.184550 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0194891c-e31d-430a-a3bb-758816656213 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:40.184777 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0194891c-e31d-430a-a3bb-758816656213 None admin] RBAC: Adding query filter params (interface=public, service_id=ef3a245f1f14414b9563aa30975508ad, region_id=RegionOne) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:40.185008 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0194891c-e31d-430a-a3bb-758816656213 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:40.187772 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0194891c-e31d-430a-a3bb-758816656213 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:40.187986 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f16a2d-f237-4df8-ad6e-83c49af29d67 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:40.196865 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 346/696] 10.222.0.59 () {56 vars in 1251 bytes} [Tue Oct 11 11:47:39 2022] GET /identity/v3/endpoints?service_id=ef3a245f1f14414b9563aa30975508ad&interface=public®ion_id=RegionOne => generated 194 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:47:40.205109 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0194891c-e31d-430a-a3bb-758816656213 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:40.298727 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f16a2d-f237-4df8-ad6e-83c49af29d67 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:40.309209 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0194891c-e31d-430a-a3bb-758816656213 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:40.336530 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f36bdd-8a46-4df1-9829-3e818de4ce85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:40.337366 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f36bdd-8a46-4df1-9829-3e818de4ce85 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:40.337709 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f36bdd-8a46-4df1-9829-3e818de4ce85 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:40.338067 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f36bdd-8a46-4df1-9829-3e818de4ce85 None admin] PATH_INFO: `/v3/services` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:40.338873 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f36bdd-8a46-4df1-9829-3e818de4ce85 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:40.339236 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f36bdd-8a46-4df1-9829-3e818de4ce85 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:40.340848 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f36bdd-8a46-4df1-9829-3e818de4ce85 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:40.347637 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 351/697] 10.222.0.59 () {56 vars in 1122 bytes} [Tue Oct 11 11:47:40 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:47:40.348653 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca4eb16-b9bf-446c-b59c-b417e2faeab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:40.349287 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca4eb16-b9bf-446c-b59c-b417e2faeab3 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:40.349660 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca4eb16-b9bf-446c-b59c-b417e2faeab3 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:40.349934 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca4eb16-b9bf-446c-b59c-b417e2faeab3 None admin] PATH_INFO: `/v3/services` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:40.350550 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca4eb16-b9bf-446c-b59c-b417e2faeab3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:40.351949 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca4eb16-b9bf-446c-b59c-b417e2faeab3 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:40.358713 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f36bdd-8a46-4df1-9829-3e818de4ce85 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:40.361134 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 347/698] 10.222.0.59 () {56 vars in 1087 bytes} [Tue Oct 11 11:47:40 2022] GET /identity/v3/services => generated 1497 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 11:47:40.468182 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:40.552942 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f36bdd-8a46-4df1-9829-3e818de4ce85 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:40.590063 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d69eff-2926-4653-9d36-e69eaeba351d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:40.591423 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d69eff-2926-4653-9d36-e69eaeba351d None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:40.592144 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d69eff-2926-4653-9d36-e69eaeba351d None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:40.592833 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d69eff-2926-4653-9d36-e69eaeba351d None admin] PATH_INFO: `/v3/services` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:40.594252 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d69eff-2926-4653-9d36-e69eaeba351d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:40.595132 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d69eff-2926-4653-9d36-e69eaeba351d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:40.597965 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d69eff-2926-4653-9d36-e69eaeba351d None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:40.606388 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 352/699] 10.222.0.59 () {56 vars in 1122 bytes} [Tue Oct 11 11:47:40 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:47:40.612820 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca4eb16-b9bf-446c-b59c-b417e2faeab3 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:40.648578 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:40.730310 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca4eb16-b9bf-446c-b59c-b417e2faeab3 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:40.768039 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e26be92-99d2-4ae6-ac3f-2c714bbcf192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:40.769409 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e26be92-99d2-4ae6-ac3f-2c714bbcf192 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:40.769850 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e26be92-99d2-4ae6-ac3f-2c714bbcf192 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:40.770273 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e26be92-99d2-4ae6-ac3f-2c714bbcf192 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:40.771165 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e26be92-99d2-4ae6-ac3f-2c714bbcf192 None admin] RBAC: Adding query filter params (interface=public, service_id=3022788117c44b9682bcfe86a7d31099, region_id=RegionOne) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:40.771657 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e26be92-99d2-4ae6-ac3f-2c714bbcf192 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:40.773198 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e26be92-99d2-4ae6-ac3f-2c714bbcf192 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:40.779625 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 348/700] 10.222.0.59 () {56 vars in 1251 bytes} [Tue Oct 11 11:47:40 2022] GET /identity/v3/endpoints?service_id=3022788117c44b9682bcfe86a7d31099&interface=public®ion_id=RegionOne => generated 194 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:47:40.786999 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d69eff-2926-4653-9d36-e69eaeba351d None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:40.950797 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d69eff-2926-4653-9d36-e69eaeba351d None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:40.973523 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:40.991334 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34722bf-dbc7-4a84-bc75-c2fbbfa2381f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:40.992002 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34722bf-dbc7-4a84-bc75-c2fbbfa2381f None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:40.992283 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34722bf-dbc7-4a84-bc75-c2fbbfa2381f None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:40.992562 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34722bf-dbc7-4a84-bc75-c2fbbfa2381f None admin] PATH_INFO: `/v3/services` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:40.993282 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34722bf-dbc7-4a84-bc75-c2fbbfa2381f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:40.994799 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34722bf-dbc7-4a84-bc75-c2fbbfa2381f None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:41.007301 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 353/701] 10.222.0.59 () {56 vars in 1087 bytes} [Tue Oct 11 11:47:40 2022] GET /identity/v3/services => generated 1497 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 11 11:47:41.314636 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ac81b9-39b1-4be3-baff-41426185ea0f None None] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:41.315638 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ac81b9-39b1-4be3-baff-41426185ea0f None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:41.316377 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ac81b9-39b1-4be3-baff-41426185ea0f None None] PATH_INFO: `/` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:41.319390 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 349/702] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:47:41 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:47:41.328929 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75983833-c981-4530-ab58-77c3ed18e525 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:41.329859 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75983833-c981-4530-ab58-77c3ed18e525 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:41.330634 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75983833-c981-4530-ab58-77c3ed18e525 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:41.363214 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-75983833-c981-4530-ab58-77c3ed18e525 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:47:41.365964 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-75983833-c981-4530-ab58-77c3ed18e525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:47:41.457000 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 354/703] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:47:41 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 11:47:41.806334 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c6e693-89d9-4fb0-abb4-9b500ecd341b None None] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:41.807767 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c6e693-89d9-4fb0-abb4-9b500ecd341b None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:41.808576 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c6e693-89d9-4fb0-abb4-9b500ecd341b None None] PATH_INFO: `/` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:41.812000 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 350/704] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:47:41 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:47:41.817773 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86933d3b-968b-46d4-8b66-e36dfc381aab None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:41.818590 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86933d3b-968b-46d4-8b66-e36dfc381aab None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:41.818590 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86933d3b-968b-46d4-8b66-e36dfc381aab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:41.838790 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-86933d3b-968b-46d4-8b66-e36dfc381aab None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:47:41.840024 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-86933d3b-968b-46d4-8b66-e36dfc381aab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:47:41.924534 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 355/705] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:47:41 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 11 11:47:41.957279 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c6e693-89d9-4fb0-abb4-9b500ecd341b None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:42.127015 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:42.127841 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c6e693-89d9-4fb0-abb4-9b500ecd341b None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:42.165652 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07356cb6-302d-4246-90dc-88d7fe7b0464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:42.166302 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07356cb6-302d-4246-90dc-88d7fe7b0464 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:42.166608 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07356cb6-302d-4246-90dc-88d7fe7b0464 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:42.166884 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07356cb6-302d-4246-90dc-88d7fe7b0464 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:42.167479 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07356cb6-302d-4246-90dc-88d7fe7b0464 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:42.168894 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07356cb6-302d-4246-90dc-88d7fe7b0464 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:42.178863 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application [None req-07356cb6-302d-4246-90dc-88d7fe7b0464 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 11 11:47:42.178863 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:47:42.178863 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 11 11:47:42.178863 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 11 11:47:42.178863 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 11 11:47:42.178863 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 11 11:47:42.178863 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 11 11:47:42.178863 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 11:47:42.178863 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 11 11:47:42.178863 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:47:42.178863 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 11 11:47:42.178863 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:47:42.178863 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:47:42.178863 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:47:42.178863 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:47:42.178863 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:47:42.180644 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:47:42.180644 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:47:42.180644 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:47:42.180644 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:47:42.180644 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:47:42.180644 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:47:42.180644 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:47:42.180644 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 11 11:47:42.180644 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 11 11:47:42.180644 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 11 11:47:42.180644 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 11 11:47:42.180644 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:47:42.180644 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:47:42.180644 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:47:42.180644 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:47:42.180644 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:47:42.180644 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:47:42.182389 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:47:42.182389 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application with Lock( Oct 11 11:47:42.182389 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:47:42.182389 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:47:42.182389 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:47:42.182389 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:47:42.182389 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:47:42.182389 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:47:42.182389 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:47:42.182389 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:47:42.182389 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 11 11:47:42.182389 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 11:47:42.182389 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 11 11:47:42.182389 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:47:42.182389 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 351/706] 10.222.0.59 () {56 vars in 1132 bytes} [Tue Oct 11 11:47:41 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:47:42.195923 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86933d3b-968b-46d4-8b66-e36dfc381aab None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:42.249234 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07356cb6-302d-4246-90dc-88d7fe7b0464 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:42.264304 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:42.342494 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86933d3b-968b-46d4-8b66-e36dfc381aab None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:42.395542 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed412ac-140f-44d6-a8a1-7c363f6a4c6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:42.396659 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed412ac-140f-44d6-a8a1-7c363f6a4c6d None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:42.397147 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed412ac-140f-44d6-a8a1-7c363f6a4c6d None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:42.397755 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed412ac-140f-44d6-a8a1-7c363f6a4c6d None admin] PATH_INFO: `/v3/services` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:42.398930 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed412ac-140f-44d6-a8a1-7c363f6a4c6d None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:42.399469 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed412ac-140f-44d6-a8a1-7c363f6a4c6d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:42.402351 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed412ac-140f-44d6-a8a1-7c363f6a4c6d None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:42.403471 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07356cb6-302d-4246-90dc-88d7fe7b0464 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:42.419156 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 356/707] 10.222.0.59 () {56 vars in 1126 bytes} [Tue Oct 11 11:47:42 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:47:42.422524 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed412ac-140f-44d6-a8a1-7c363f6a4c6d None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:42.443363 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027f8982-78b5-4628-9255-ab1b7ca40d9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:42.443995 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027f8982-78b5-4628-9255-ab1b7ca40d9f None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:42.444264 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027f8982-78b5-4628-9255-ab1b7ca40d9f None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:42.444540 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027f8982-78b5-4628-9255-ab1b7ca40d9f None admin] PATH_INFO: `/v3/services/image` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:42.445145 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027f8982-78b5-4628-9255-ab1b7ca40d9f None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:42.448105 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027f8982-78b5-4628-9255-ab1b7ca40d9f None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:42.458630 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application [None req-027f8982-78b5-4628-9255-ab1b7ca40d9f None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 11 11:47:42.458630 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:47:42.458630 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 11 11:47:42.458630 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 11 11:47:42.458630 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 11 11:47:42.458630 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 11 11:47:42.458630 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 11 11:47:42.458630 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 11:47:42.458630 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 11 11:47:42.458630 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:47:42.458630 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 11 11:47:42.458630 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:47:42.458630 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:47:42.458630 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:47:42.458630 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:47:42.458630 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:47:42.458630 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:47:42.460628 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:47:42.460628 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:47:42.460628 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:47:42.460628 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:47:42.460628 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:47:42.460628 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:47:42.460628 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 11 11:47:42.460628 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 11 11:47:42.460628 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 11 11:47:42.460628 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 11 11:47:42.460628 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:47:42.460628 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:47:42.460628 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:47:42.460628 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:47:42.460628 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:47:42.460628 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:47:42.462170 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:47:42.462170 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application with Lock( Oct 11 11:47:42.462170 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:47:42.462170 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:47:42.462170 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:47:42.462170 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:47:42.462170 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:47:42.462170 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:47:42.462170 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:47:42.462170 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:47:42.462170 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 11 11:47:42.462170 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 11:47:42.462170 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 11 11:47:42.462170 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:47:42.462170 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 352/708] 10.222.0.59 () {56 vars in 1105 bytes} [Tue Oct 11 11:47:42 2022] GET /identity/v3/services/image => generated 86 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 11 11:47:42.468361 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027f8982-78b5-4628-9255-ab1b7ca40d9f None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:42.515633 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:42.592724 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed412ac-140f-44d6-a8a1-7c363f6a4c6d None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:42.595347 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:42.633953 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027f8982-78b5-4628-9255-ab1b7ca40d9f None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:42.643777 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb6a9da-c8e6-420b-8b7e-886cdcc8b101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:42.645142 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb6a9da-c8e6-420b-8b7e-886cdcc8b101 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:42.645690 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb6a9da-c8e6-420b-8b7e-886cdcc8b101 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:42.646433 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb6a9da-c8e6-420b-8b7e-886cdcc8b101 None admin] PATH_INFO: `/v3/services` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:42.647545 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb6a9da-c8e6-420b-8b7e-886cdcc8b101 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:42.648005 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb6a9da-c8e6-420b-8b7e-886cdcc8b101 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:42.649731 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb6a9da-c8e6-420b-8b7e-886cdcc8b101 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:42.661936 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 357/709] 10.222.0.59 () {56 vars in 1126 bytes} [Tue Oct 11 11:47:42 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 11:47:42.666106 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb6a9da-c8e6-420b-8b7e-886cdcc8b101 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:42.683049 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5468bbc5-f12a-4112-9d0a-829a77460080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:42.684678 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5468bbc5-f12a-4112-9d0a-829a77460080 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:42.685238 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5468bbc5-f12a-4112-9d0a-829a77460080 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:42.685707 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5468bbc5-f12a-4112-9d0a-829a77460080 None admin] PATH_INFO: `/v3/services` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:42.687129 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5468bbc5-f12a-4112-9d0a-829a77460080 None admin] RBAC: Adding query filter params (name=image) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:42.687994 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5468bbc5-f12a-4112-9d0a-829a77460080 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:42.689778 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5468bbc5-f12a-4112-9d0a-829a77460080 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:42.697207 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 353/710] 10.222.0.59 () {56 vars in 1108 bytes} [Tue Oct 11 11:47:42 2022] GET /identity/v3/services?name=image => generated 122 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:47:42.704665 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5468bbc5-f12a-4112-9d0a-829a77460080 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:42.804433 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb6a9da-c8e6-420b-8b7e-886cdcc8b101 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:42.842925 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1ed06a-5113-469b-90e0-c1c195a1c1c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:42.844639 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1ed06a-5113-469b-90e0-c1c195a1c1c5 None admin] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:42.845146 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1ed06a-5113-469b-90e0-c1c195a1c1c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:42.845618 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1ed06a-5113-469b-90e0-c1c195a1c1c5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:42.846542 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1ed06a-5113-469b-90e0-c1c195a1c1c5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:42.848201 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1ed06a-5113-469b-90e0-c1c195a1c1c5 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:42.881267 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:42.883726 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 358/711] 10.222.0.59 () {60 vars in 1144 bytes} [Tue Oct 11 11:47:42 2022] POST /identity/v3/endpoints => generated 347 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 11:47:42.904817 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5468bbc5-f12a-4112-9d0a-829a77460080 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:42.987513 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52824030-0f0f-41a7-9747-5f48ffcf130b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:42.988555 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52824030-0f0f-41a7-9747-5f48ffcf130b None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:42.988983 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52824030-0f0f-41a7-9747-5f48ffcf130b None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:42.989431 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52824030-0f0f-41a7-9747-5f48ffcf130b None admin] PATH_INFO: `/v3/services` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:42.990549 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52824030-0f0f-41a7-9747-5f48ffcf130b None admin] RBAC: Adding query filter params (type=image) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:42.991277 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52824030-0f0f-41a7-9747-5f48ffcf130b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:42.993826 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52824030-0f0f-41a7-9747-5f48ffcf130b None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:43.007563 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 354/712] 10.222.0.59 () {56 vars in 1108 bytes} [Tue Oct 11 11:47:42 2022] GET /identity/v3/services?type=image => generated 122 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 11:47:43.176862 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7614f1df-d41b-4e80-84a5-ab922176c2bc None None] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:43.177234 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7614f1df-d41b-4e80-84a5-ab922176c2bc None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:43.177646 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7614f1df-d41b-4e80-84a5-ab922176c2bc None None] PATH_INFO: `/` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:43.179495 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 359/713] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:47:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:47:43.186474 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06842e20-aa62-47c8-9d9f-df81f6f0f350 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:43.187158 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06842e20-aa62-47c8-9d9f-df81f6f0f350 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:43.187631 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06842e20-aa62-47c8-9d9f-df81f6f0f350 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:43.208477 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-06842e20-aa62-47c8-9d9f-df81f6f0f350 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:47:43.210146 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-06842e20-aa62-47c8-9d9f-df81f6f0f350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:47:43.300486 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 355/714] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:47:43 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 11:47:43.363084 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b3aa9d-e9f7-4622-889d-9b4128990ce0 None None] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:43.363373 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b3aa9d-e9f7-4622-889d-9b4128990ce0 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:43.363642 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b3aa9d-e9f7-4622-889d-9b4128990ce0 None None] PATH_INFO: `/` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:43.365384 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 360/715] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:47:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:47:43.371456 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f7a324-5873-4cbe-95ef-319a4689268e None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:43.371775 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f7a324-5873-4cbe-95ef-319a4689268e None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:43.372013 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f7a324-5873-4cbe-95ef-319a4689268e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:43.399459 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-04f7a324-5873-4cbe-95ef-319a4689268e None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:47:43.401376 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-04f7a324-5873-4cbe-95ef-319a4689268e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:47:43.488727 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 356/716] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:47:43 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 11:47:43.722219 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b3aa9d-e9f7-4622-889d-9b4128990ce0 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:43.757053 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:43.813119 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f7a324-5873-4cbe-95ef-319a4689268e None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:43.834367 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:43.849316 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b3aa9d-e9f7-4622-889d-9b4128990ce0 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:43.890421 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9791f10e-4858-4a63-9034-7fd2f7cbc99a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:43.891119 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9791f10e-4858-4a63-9034-7fd2f7cbc99a None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:43.891476 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9791f10e-4858-4a63-9034-7fd2f7cbc99a None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:43.891802 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9791f10e-4858-4a63-9034-7fd2f7cbc99a None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:43.892401 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9791f10e-4858-4a63-9034-7fd2f7cbc99a None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:43.893887 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9791f10e-4858-4a63-9034-7fd2f7cbc99a None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:43.903922 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application [None req-9791f10e-4858-4a63-9034-7fd2f7cbc99a None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 11 11:47:43.903922 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:47:43.903922 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 11 11:47:43.903922 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 11 11:47:43.903922 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 11 11:47:43.903922 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 11 11:47:43.903922 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 11 11:47:43.903922 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 11:47:43.903922 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 11 11:47:43.903922 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:47:43.903922 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 11 11:47:43.903922 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:47:43.903922 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:47:43.903922 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:47:43.903922 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:47:43.903922 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:47:43.903922 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:47:43.911317 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:47:43.911317 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:47:43.911317 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:47:43.911317 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:47:43.911317 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:47:43.911317 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:47:43.911317 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 11 11:47:43.911317 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 11 11:47:43.911317 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 11 11:47:43.911317 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 11 11:47:43.911317 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:47:43.911317 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:47:43.911317 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:47:43.911317 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:47:43.911317 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:47:43.911317 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:47:43.912157 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:47:43.912157 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application with Lock( Oct 11 11:47:43.912157 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:47:43.912157 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:47:43.912157 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:47:43.912157 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:47:43.912157 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:47:43.912157 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:47:43.912157 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:47:43.912157 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:47:43.912157 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 11 11:47:43.912157 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 11:47:43.912157 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 11 11:47:43.912157 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:47:43.912157 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 361/717] 10.222.0.59 () {56 vars in 1114 bytes} [Tue Oct 11 11:47:43 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:47:43.916233 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9791f10e-4858-4a63-9034-7fd2f7cbc99a None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:44.039742 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:44.040036 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f7a324-5873-4cbe-95ef-319a4689268e None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:44.059376 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9791f10e-4858-4a63-9034-7fd2f7cbc99a None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:44.099509 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0645e985-af81-4e02-9dc6-718853ba8904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:44.100543 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0645e985-af81-4e02-9dc6-718853ba8904 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:44.101039 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0645e985-af81-4e02-9dc6-718853ba8904 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:44.101534 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0645e985-af81-4e02-9dc6-718853ba8904 None admin] PATH_INFO: `/v3/services` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:44.102628 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0645e985-af81-4e02-9dc6-718853ba8904 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:44.103174 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0645e985-af81-4e02-9dc6-718853ba8904 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:44.105713 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0645e985-af81-4e02-9dc6-718853ba8904 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:44.112230 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64ef289-58bf-447d-a431-d6fb6cc9865a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:44.112902 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 362/718] 10.222.0.59 () {56 vars in 1114 bytes} [Tue Oct 11 11:47:43 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 11:47:44.119968 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64ef289-58bf-447d-a431-d6fb6cc9865a None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:44.120455 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64ef289-58bf-447d-a431-d6fb6cc9865a None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:44.120951 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64ef289-58bf-447d-a431-d6fb6cc9865a None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:44.122098 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64ef289-58bf-447d-a431-d6fb6cc9865a None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:44.125052 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64ef289-58bf-447d-a431-d6fb6cc9865a None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:44.129089 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0645e985-af81-4e02-9dc6-718853ba8904 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:44.146037 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application [None req-f64ef289-58bf-447d-a431-d6fb6cc9865a None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 11 11:47:44.146037 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:47:44.146037 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 11 11:47:44.146037 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 11 11:47:44.146037 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 11 11:47:44.146037 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 11 11:47:44.146037 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 11 11:47:44.146037 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 11:47:44.146037 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 11 11:47:44.146037 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:47:44.146037 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 11 11:47:44.146037 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:47:44.146037 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:47:44.146037 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:47:44.146037 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:47:44.146037 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:47:44.146037 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:47:44.147293 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:47:44.147293 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:47:44.147293 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:47:44.147293 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:47:44.147293 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:47:44.147293 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:47:44.147293 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 11 11:47:44.147293 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 11 11:47:44.147293 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 11 11:47:44.147293 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 11 11:47:44.147293 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:47:44.147293 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:47:44.147293 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:47:44.147293 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:47:44.147293 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:47:44.147293 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:47:44.148226 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:47:44.148226 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application with Lock( Oct 11 11:47:44.148226 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:47:44.148226 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:47:44.148226 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:47:44.148226 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:47:44.148226 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:47:44.148226 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:47:44.148226 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:47:44.148226 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:47:44.148226 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 11 11:47:44.148226 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 11:47:44.148226 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 11 11:47:44.148226 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:47:44.155236 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 357/719] 10.222.0.59 () {56 vars in 1126 bytes} [Tue Oct 11 11:47:43 2022] GET /identity/v3/services/object-store => generated 93 bytes in 343 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:47:44.160012 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64ef289-58bf-447d-a431-d6fb6cc9865a None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:44.250433 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0645e985-af81-4e02-9dc6-718853ba8904 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:44.289034 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb992dab-f021-4775-8625-2a7ce8978886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:44.290163 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb992dab-f021-4775-8625-2a7ce8978886 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:44.290697 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb992dab-f021-4775-8625-2a7ce8978886 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:44.291186 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb992dab-f021-4775-8625-2a7ce8978886 None admin] PATH_INFO: `/v3/services` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:44.292111 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb992dab-f021-4775-8625-2a7ce8978886 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:44.292647 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb992dab-f021-4775-8625-2a7ce8978886 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:44.294375 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb992dab-f021-4775-8625-2a7ce8978886 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:44.301063 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 363/720] 10.222.0.59 () {56 vars in 1114 bytes} [Tue Oct 11 11:47:44 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 11:47:44.310389 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb992dab-f021-4775-8625-2a7ce8978886 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:44.316414 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:44.347745 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64ef289-58bf-447d-a431-d6fb6cc9865a None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:44.370994 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:44.393283 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0e65af-6ee9-4778-a13f-541ea59c8811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:44.394039 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0e65af-6ee9-4778-a13f-541ea59c8811 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:44.394394 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0e65af-6ee9-4778-a13f-541ea59c8811 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:44.394755 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0e65af-6ee9-4778-a13f-541ea59c8811 None admin] PATH_INFO: `/v3/services` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:44.395690 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0e65af-6ee9-4778-a13f-541ea59c8811 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:44.396098 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0e65af-6ee9-4778-a13f-541ea59c8811 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:44.397698 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0e65af-6ee9-4778-a13f-541ea59c8811 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:44.414384 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 358/721] 10.222.0.59 () {56 vars in 1122 bytes} [Tue Oct 11 11:47:44 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:47:44.416567 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0e65af-6ee9-4778-a13f-541ea59c8811 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:44.474570 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb992dab-f021-4775-8625-2a7ce8978886 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:44.512645 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bbe52f-13df-4a98-b248-452d0de3df60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:44.513271 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bbe52f-13df-4a98-b248-452d0de3df60 None admin] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:44.513550 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bbe52f-13df-4a98-b248-452d0de3df60 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:44.513816 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bbe52f-13df-4a98-b248-452d0de3df60 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:44.514513 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bbe52f-13df-4a98-b248-452d0de3df60 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:44.515921 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bbe52f-13df-4a98-b248-452d0de3df60 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:44.550697 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 364/722] 10.222.0.59 () {60 vars in 1144 bytes} [Tue Oct 11 11:47:44 2022] POST /identity/v3/endpoints => generated 346 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 11:47:44.596375 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0e65af-6ee9-4778-a13f-541ea59c8811 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:44.659786 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9088e1da-de49-45d5-883e-f271c9989fd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:44.661390 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9088e1da-de49-45d5-883e-f271c9989fd7 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:44.662206 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9088e1da-de49-45d5-883e-f271c9989fd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:44.663050 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9088e1da-de49-45d5-883e-f271c9989fd7 None admin] PATH_INFO: `/v3/services` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:44.664569 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9088e1da-de49-45d5-883e-f271c9989fd7 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:44.665446 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9088e1da-de49-45d5-883e-f271c9989fd7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:44.668347 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9088e1da-de49-45d5-883e-f271c9989fd7 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:44.678982 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:44.683158 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 359/723] 10.222.0.59 () {56 vars in 1122 bytes} [Tue Oct 11 11:47:44 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 11:47:44.691733 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bbe52f-13df-4a98-b248-452d0de3df60 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:44.829136 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:44.889202 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bbe52f-13df-4a98-b248-452d0de3df60 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:44.966696 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca496c6-7d2b-47e4-b939-b8755e4bf7ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:44.967628 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca496c6-7d2b-47e4-b939-b8755e4bf7ef None admin] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:44.968111 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca496c6-7d2b-47e4-b939-b8755e4bf7ef None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:44.968604 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca496c6-7d2b-47e4-b939-b8755e4bf7ef None admin] PATH_INFO: `/v3/endpoints` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:44.969593 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca496c6-7d2b-47e4-b939-b8755e4bf7ef None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:44.971371 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca496c6-7d2b-47e4-b939-b8755e4bf7ef None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:44.996940 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 365/724] 10.222.0.59 () {60 vars in 1144 bytes} [Tue Oct 11 11:47:44 2022] POST /identity/v3/endpoints => generated 349 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 11:47:45.249182 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e213719-11e8-4bad-bdb4-c2f7094366bc None None] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:45.249714 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e213719-11e8-4bad-bdb4-c2f7094366bc None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:45.249714 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e213719-11e8-4bad-bdb4-c2f7094366bc None None] PATH_INFO: `/` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:45.251416 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 360/725] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:47:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:47:45.257426 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1466d8f3-c630-4d18-b7d0-304946bf28d9 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:45.257736 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1466d8f3-c630-4d18-b7d0-304946bf28d9 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:45.257988 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1466d8f3-c630-4d18-b7d0-304946bf28d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:45.278206 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-1466d8f3-c630-4d18-b7d0-304946bf28d9 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:47:45.279469 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-1466d8f3-c630-4d18-b7d0-304946bf28d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:47:45.367320 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 366/726] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:47:45 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 11:47:45.379805 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc6597d-d671-46fb-aa90-83dadb122012 None None] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:45.380290 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc6597d-d671-46fb-aa90-83dadb122012 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:45.380790 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc6597d-d671-46fb-aa90-83dadb122012 None None] PATH_INFO: `/` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:45.385570 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 361/727] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:47:45 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:47:45.393288 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8061e62-a1d9-40d0-a822-9aed818ce721 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:45.393773 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8061e62-a1d9-40d0-a822-9aed818ce721 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:45.394221 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8061e62-a1d9-40d0-a822-9aed818ce721 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:45.425228 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-a8061e62-a1d9-40d0-a822-9aed818ce721 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:47:45.427371 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-a8061e62-a1d9-40d0-a822-9aed818ce721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:47:45.487804 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:45.510874 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 367/728] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:47:45 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 11:47:45.634179 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc6597d-d671-46fb-aa90-83dadb122012 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:45.836214 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8061e62-a1d9-40d0-a822-9aed818ce721 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:45.850588 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc6597d-d671-46fb-aa90-83dadb122012 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:45.911238 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d073eb9-368c-4390-8d30-61fea8965b64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:45.912352 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d073eb9-368c-4390-8d30-61fea8965b64 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:45.912853 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d073eb9-368c-4390-8d30-61fea8965b64 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:45.913351 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d073eb9-368c-4390-8d30-61fea8965b64 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:45.914473 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d073eb9-368c-4390-8d30-61fea8965b64 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:45.917259 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d073eb9-368c-4390-8d30-61fea8965b64 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:45.935966 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application [None req-5d073eb9-368c-4390-8d30-61fea8965b64 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 11 11:47:45.935966 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:47:45.935966 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 11 11:47:45.935966 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 11 11:47:45.935966 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 11 11:47:45.935966 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 11 11:47:45.935966 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 11 11:47:45.935966 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 11:47:45.935966 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 11 11:47:45.935966 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:47:45.935966 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 11 11:47:45.935966 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:47:45.935966 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:47:45.935966 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:47:45.935966 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:47:45.935966 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:47:45.935966 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:47:45.937138 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:47:45.937138 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:47:45.937138 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:47:45.937138 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:47:45.937138 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:47:45.937138 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:47:45.937138 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 11 11:47:45.937138 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 11 11:47:45.937138 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 11 11:47:45.937138 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 11 11:47:45.937138 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:47:45.937138 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:47:45.937138 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:47:45.937138 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:47:45.937138 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:47:45.937138 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:47:45.937981 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:47:45.937981 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application with Lock( Oct 11 11:47:45.937981 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:47:45.937981 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:47:45.937981 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:47:45.937981 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:47:45.937981 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:47:45.937981 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:47:45.937981 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:47:45.937981 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:47:45.937981 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 11 11:47:45.937981 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 11:47:45.937981 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 11 11:47:45.937981 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:47:45.939094 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 362/729] 10.222.0.59 () {56 vars in 1111 bytes} [Tue Oct 11 11:47:45 2022] GET /identity/v3/services/compute => generated 88 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:47:45.945243 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d073eb9-368c-4390-8d30-61fea8965b64 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:45.952402 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8061e62-a1d9-40d0-a822-9aed818ce721 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:46.000891 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df12d24-1e5c-4a0f-94fd-399db8904e26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:46.002241 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df12d24-1e5c-4a0f-94fd-399db8904e26 None admin] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:46.002739 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df12d24-1e5c-4a0f-94fd-399db8904e26 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:46.003518 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df12d24-1e5c-4a0f-94fd-399db8904e26 None admin] PATH_INFO: `/v3/services` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:46.004676 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df12d24-1e5c-4a0f-94fd-399db8904e26 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:46.006501 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df12d24-1e5c-4a0f-94fd-399db8904e26 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:46.034602 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 368/730] 10.222.0.59 () {60 vars in 1141 bytes} [Tue Oct 11 11:47:45 2022] POST /identity/v3/services => generated 242 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 11:47:46.034859 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:46.085510 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d073eb9-368c-4390-8d30-61fea8965b64 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:46.137958 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac369f9-f017-487c-bb3b-e515c469ab30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:46.138585 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac369f9-f017-487c-bb3b-e515c469ab30 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:46.138884 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac369f9-f017-487c-bb3b-e515c469ab30 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:46.139157 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac369f9-f017-487c-bb3b-e515c469ab30 None admin] PATH_INFO: `/v3/services` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:46.139868 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac369f9-f017-487c-bb3b-e515c469ab30 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:46.140174 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac369f9-f017-487c-bb3b-e515c469ab30 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:46.141685 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac369f9-f017-487c-bb3b-e515c469ab30 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:46.148279 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 363/731] 10.222.0.59 () {56 vars in 1112 bytes} [Tue Oct 11 11:47:45 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:47:46.155820 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df12d24-1e5c-4a0f-94fd-399db8904e26 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:46.209830 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:46.276282 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df12d24-1e5c-4a0f-94fd-399db8904e26 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:46.313457 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad2b9d4-47e0-45b4-ad20-14de63aec048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:46.314432 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad2b9d4-47e0-45b4-ad20-14de63aec048 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:46.314964 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad2b9d4-47e0-45b4-ad20-14de63aec048 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:46.315432 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad2b9d4-47e0-45b4-ad20-14de63aec048 None admin] PATH_INFO: `/v3/services` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:46.316340 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad2b9d4-47e0-45b4-ad20-14de63aec048 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:46.316838 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad2b9d4-47e0-45b4-ad20-14de63aec048 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:46.318430 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad2b9d4-47e0-45b4-ad20-14de63aec048 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:46.324366 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 369/732] 10.222.0.59 () {56 vars in 1112 bytes} [Tue Oct 11 11:47:46 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:47:47.202954 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4909b2-2d9b-401a-9ae8-3bbccbd71065 None None] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:47.204562 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4909b2-2d9b-401a-9ae8-3bbccbd71065 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:47.205264 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4909b2-2d9b-401a-9ae8-3bbccbd71065 None None] PATH_INFO: `/` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:47.208220 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 364/733] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:47:47 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:47:47.216460 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632a4202-bb1d-47ef-abc7-d05df295191b None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:47.217179 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632a4202-bb1d-47ef-abc7-d05df295191b None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:47.217179 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632a4202-bb1d-47ef-abc7-d05df295191b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:47.237627 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-632a4202-bb1d-47ef-abc7-d05df295191b None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:47:47.238867 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-632a4202-bb1d-47ef-abc7-d05df295191b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:47:47.313735 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 370/734] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:47:47 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 11:47:47.817644 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4909b2-2d9b-401a-9ae8-3bbccbd71065 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:47.994300 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4909b2-2d9b-401a-9ae8-3bbccbd71065 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:48.020551 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:48.033961 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfb2487-3b21-4251-aed0-3ad1bf1e45ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:48.034929 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfb2487-3b21-4251-aed0-3ad1bf1e45ab None admin] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:48.035209 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfb2487-3b21-4251-aed0-3ad1bf1e45ab None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:48.035487 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfb2487-3b21-4251-aed0-3ad1bf1e45ab None admin] PATH_INFO: `/v3/projects` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:48.036147 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfb2487-3b21-4251-aed0-3ad1bf1e45ab None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:48.037593 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfb2487-3b21-4251-aed0-3ad1bf1e45ab None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:48.040366 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: WARNING keystone.server.flask.common [None req-bdfb2487-3b21-4251-aed0-3ad1bf1e45ab None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Oct 11 11:47:48.061240 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.sql.core [None req-bdfb2487-3b21-4251-aed0-3ad1bf1e45ab None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 11 11:47:48.061240 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 11 11:47:48.061240 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [parameters: {'id': 'f20d77d4fe1040ddbf149d3a938682e5', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 11 11:47:48.061240 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81433) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 11 11:47:48.069499 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application [None req-bdfb2487-3b21-4251-aed0-3ad1bf1e45ab None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id f20d77d4fe1040ddbf149d3a938682e5.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id f20d77d4fe1040ddbf149d3a938682e5. Oct 11 11:47:48.069499 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:47:48.069499 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 11 11:47:48.069499 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 11 11:47:48.069499 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 11 11:47:48.069499 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 11 11:47:48.069499 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 11 11:47:48.069499 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application result = self._query(query) Oct 11 11:47:48.069499 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 11 11:47:48.069499 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application conn.query(q) Oct 11 11:47:48.069499 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 11 11:47:48.069499 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 11 11:47:48.069499 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 11 11:47:48.071386 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application result.read() Oct 11 11:47:48.071386 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 11 11:47:48.071386 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 11 11:47:48.071386 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 11 11:47:48.071386 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 11 11:47:48.071386 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 11 11:47:48.071386 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 11 11:47:48.071386 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 11 11:47:48.071386 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 11 11:47:48.071386 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 11 11:47:48.071386 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:47:48.071386 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 11 11:47:48.071386 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:47:48.071386 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:47:48.071386 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 11 11:47:48.071386 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 11 11:47:48.071386 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 11 11:47:48.071386 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 11 11:47:48.073757 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 11 11:47:48.073757 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application next(self.gen) Oct 11 11:47:48.073757 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 11 11:47:48.073757 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application yield resource Oct 11 11:47:48.073757 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 11 11:47:48.073757 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application next(self.gen) Oct 11 11:47:48.073757 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 11 11:47:48.073757 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application self.session.rollback() Oct 11 11:47:48.073757 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 11 11:47:48.073757 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application self.force_reraise() Oct 11 11:47:48.073757 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 11 11:47:48.073757 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application raise self.value Oct 11 11:47:48.073757 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 11 11:47:48.073757 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 11 11:47:48.073757 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 11 11:47:48.073757 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application session.commit() Oct 11 11:47:48.073757 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 11 11:47:48.075624 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 11 11:47:48.075624 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 11 11:47:48.075624 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application self._prepare_impl() Oct 11 11:47:48.075624 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 11 11:47:48.075624 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application self.session.flush() Oct 11 11:47:48.075624 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 11 11:47:48.075624 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application self._flush(objects) Oct 11 11:47:48.075624 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 11 11:47:48.075624 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 11 11:47:48.075624 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 11 11:47:48.075624 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application compat.raise_( Oct 11 11:47:48.075624 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 11 11:47:48.075624 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application raise exception Oct 11 11:47:48.075624 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 11 11:47:48.075624 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application flush_context.execute() Oct 11 11:47:48.075624 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 11 11:47:48.075624 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application rec.execute(self) Oct 11 11:47:48.077359 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 11 11:47:48.077359 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 11 11:47:48.077359 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 11 11:47:48.077359 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 11 11:47:48.077359 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 11 11:47:48.077359 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 11 11:47:48.077359 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 11 11:47:48.077359 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 11 11:47:48.077359 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 11 11:47:48.077359 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 11 11:47:48.077359 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 11 11:47:48.077359 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 11 11:47:48.077359 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 11 11:47:48.077359 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 11 11:47:48.077359 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 11 11:47:48.083536 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 11 11:47:48.083536 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 11 11:47:48.083536 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application raise exception Oct 11 11:47:48.083536 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 11 11:47:48.083536 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 11 11:47:48.083536 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 11 11:47:48.083536 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 11 11:47:48.083536 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 11 11:47:48.083536 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application result = self._query(query) Oct 11 11:47:48.083536 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 11 11:47:48.083536 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application conn.query(q) Oct 11 11:47:48.083536 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 11 11:47:48.083536 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 11 11:47:48.083536 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 11 11:47:48.083536 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application result.read() Oct 11 11:47:48.083536 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 11 11:47:48.083536 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 11 11:47:48.085321 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 11 11:47:48.085321 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 11 11:47:48.085321 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 11 11:47:48.085321 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 11 11:47:48.085321 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 11 11:47:48.085321 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 11 11:47:48.085321 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 11 11:47:48.085321 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 11 11:47:48.085321 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application [parameters: {'id': 'f20d77d4fe1040ddbf149d3a938682e5', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 11 11:47:48.085321 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 11 11:47:48.085321 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:47:48.085321 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 11 11:47:48.085321 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:47:48.085321 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:47:48.085321 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 11 11:47:48.087682 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 11 11:47:48.087682 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 11 11:47:48.087682 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 11 11:47:48.087682 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Oct 11 11:47:48.087682 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:47:48.087682 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 11 11:47:48.087682 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:47:48.087682 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:47:48.087682 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:47:48.087682 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:47:48.087682 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:47:48.087682 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:47:48.087682 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:47:48.087682 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:47:48.087682 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:47:48.087682 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:47:48.087682 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:47:48.090921 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:47:48.090921 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Oct 11 11:47:48.090921 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Oct 11 11:47:48.090921 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:47:48.090921 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:47:48.090921 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 11 11:47:48.090921 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 11 11:47:48.090921 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id f20d77d4fe1040ddbf149d3a938682e5. Oct 11 11:47:48.090921 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:47:48.090921 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 365/735] 10.222.0.59 () {60 vars in 1140 bytes} [Tue Oct 11 11:47:47 2022] POST /identity/v3/projects => generated 262 bytes in 254 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:47:48.093252 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632a4202-bb1d-47ef-abc7-d05df295191b None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:48.149300 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:48.244177 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632a4202-bb1d-47ef-abc7-d05df295191b None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:48.288219 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b04db6-902f-4691-a510-89b3b3cc2b11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:48.288886 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b04db6-902f-4691-a510-89b3b3cc2b11 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:48.289183 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b04db6-902f-4691-a510-89b3b3cc2b11 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:48.289435 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b04db6-902f-4691-a510-89b3b3cc2b11 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:48.297583 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b04db6-902f-4691-a510-89b3b3cc2b11 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:48.299140 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b04db6-902f-4691-a510-89b3b3cc2b11 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:48.312110 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application [None req-c8b04db6-902f-4691-a510-89b3b3cc2b11 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 11:47:48.312110 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:47:48.312110 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:47:48.312110 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:47:48.312110 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:47:48.312110 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:47:48.312110 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:47:48.312110 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:47:48.312110 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:47:48.312110 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:47:48.312110 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:47:48.312110 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:47:48.312110 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 11 11:47:48.312110 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 11 11:47:48.312110 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 11 11:47:48.313257 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 11 11:47:48.313257 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:47:48.313257 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:47:48.313257 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:47:48.313257 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:47:48.313257 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:47:48.313257 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:47:48.313257 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:47:48.313257 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application with Lock( Oct 11 11:47:48.313257 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:47:48.313257 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:47:48.313257 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:47:48.313257 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:47:48.313257 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:47:48.313257 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:47:48.313257 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:47:48.313257 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:47:48.314243 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 11 11:47:48.314243 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 11 11:47:48.314243 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 11 11:47:48.314243 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 11 11:47:48.314243 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 11 11:47:48.314243 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 11 11:47:48.314243 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 11:47:48.314243 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:47:48.314243 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 371/736] 10.222.0.59 () {56 vars in 1111 bytes} [Tue Oct 11 11:47:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:47:48.322131 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfb2487-3b21-4251-aed0-3ad1bf1e45ab None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:48.400351 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a864cd84-b836-49a3-bd62-4e7bf951e872 None None] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:48.400622 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a864cd84-b836-49a3-bd62-4e7bf951e872 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:48.400894 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a864cd84-b836-49a3-bd62-4e7bf951e872 None None] PATH_INFO: `/` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:48.402622 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 372/737] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:47:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 11 11:47:48.412881 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0dfe2c-c5d9-4696-8004-961b59eaa102 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:48.413372 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0dfe2c-c5d9-4696-8004-961b59eaa102 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:48.413841 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0dfe2c-c5d9-4696-8004-961b59eaa102 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:48.440481 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-ce0dfe2c-c5d9-4696-8004-961b59eaa102 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:47:48.441928 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-ce0dfe2c-c5d9-4696-8004-961b59eaa102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:47:48.473435 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfb2487-3b21-4251-aed0-3ad1bf1e45ab None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:48.528269 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507ca001-6a70-441f-b7c9-46c1e216f4cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:48.529017 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507ca001-6a70-441f-b7c9-46c1e216f4cc None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:48.529357 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507ca001-6a70-441f-b7c9-46c1e216f4cc None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:48.529757 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507ca001-6a70-441f-b7c9-46c1e216f4cc None admin] PATH_INFO: `/v3/projects` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:48.530353 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507ca001-6a70-441f-b7c9-46c1e216f4cc None admin] RBAC: Adding query filter params (name=service) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:48.530713 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507ca001-6a70-441f-b7c9-46c1e216f4cc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:48.532600 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507ca001-6a70-441f-b7c9-46c1e216f4cc None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:48.535842 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 373/738] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:47:48 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 11:47:48.540849 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4194eb9-7cca-4ca0-8396-a704f3757bc1 None None] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:48.541300 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4194eb9-7cca-4ca0-8396-a704f3757bc1 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:48.541763 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4194eb9-7cca-4ca0-8396-a704f3757bc1 None None] PATH_INFO: `/` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:48.551244 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 374/739] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:47:48 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:47:48.554813 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a3f830-fa3a-4e49-88d8-ebcff18fdf8d None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:48.554813 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a3f830-fa3a-4e49-88d8-ebcff18fdf8d None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:48.555168 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a3f830-fa3a-4e49-88d8-ebcff18fdf8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:48.582271 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 366/740] 10.222.0.59 () {56 vars in 1112 bytes} [Tue Oct 11 11:47:48 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 11:47:48.590236 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-b8a3f830-fa3a-4e49-88d8-ebcff18fdf8d None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:47:48.592197 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-b8a3f830-fa3a-4e49-88d8-ebcff18fdf8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:47:48.629349 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:48.651230 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b12eae6-c6db-4e5d-8103-4702616ac67d None None] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:48.651493 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b12eae6-c6db-4e5d-8103-4702616ac67d None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:48.651833 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b12eae6-c6db-4e5d-8103-4702616ac67d None None] PATH_INFO: `/` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:48.653481 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 367/741] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:47:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 11 11:47:48.659640 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8028ca-a730-4fe4-b72a-27f0727aa408 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:48.659911 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8028ca-a730-4fe4-b72a-27f0727aa408 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:48.660239 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8028ca-a730-4fe4-b72a-27f0727aa408 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:48.684427 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-6a8028ca-a730-4fe4-b72a-27f0727aa408 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:47:48.687745 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-6a8028ca-a730-4fe4-b72a-27f0727aa408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:47:48.726783 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 375/742] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:47:48 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 11:47:48.786907 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:48.849123 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 368/743] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:47:48 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 11 11:47:48.935294 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a3f830-fa3a-4e49-88d8-ebcff18fdf8d None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:49.041559 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8028ca-a730-4fe4-b72a-27f0727aa408 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:49.062839 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a3f830-fa3a-4e49-88d8-ebcff18fdf8d None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:49.104177 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90419250-051d-47a1-87c3-1e28c7d904a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:49.104769 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90419250-051d-47a1-87c3-1e28c7d904a3 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:49.105054 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90419250-051d-47a1-87c3-1e28c7d904a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:49.105331 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90419250-051d-47a1-87c3-1e28c7d904a3 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:49.106203 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90419250-051d-47a1-87c3-1e28c7d904a3 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:49.108161 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90419250-051d-47a1-87c3-1e28c7d904a3 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:49.129475 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application [None req-90419250-051d-47a1-87c3-1e28c7d904a3 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 11 11:47:49.129475 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:47:49.129475 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 11 11:47:49.129475 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 11 11:47:49.129475 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 11 11:47:49.129475 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 11 11:47:49.129475 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 11 11:47:49.129475 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 11:47:49.129475 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 11 11:47:49.129475 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:47:49.129475 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 11 11:47:49.129475 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:47:49.129475 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:47:49.129475 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:47:49.129475 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:47:49.129475 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:47:49.129475 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:47:49.130584 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:47:49.130584 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:47:49.130584 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:47:49.130584 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:47:49.130584 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:47:49.130584 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:47:49.130584 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 11 11:47:49.130584 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 11 11:47:49.130584 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 11 11:47:49.130584 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 11 11:47:49.130584 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:47:49.130584 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:47:49.130584 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:47:49.130584 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:47:49.130584 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:47:49.130584 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:47:49.131433 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:47:49.131433 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application with Lock( Oct 11 11:47:49.131433 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:47:49.131433 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:47:49.131433 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:47:49.131433 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:47:49.131433 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:47:49.131433 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:47:49.131433 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:47:49.131433 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:47:49.131433 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 11 11:47:49.131433 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 11:47:49.131433 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 11 11:47:49.131433 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:47:49.134611 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 376/744] 10.222.0.59 () {56 vars in 1126 bytes} [Tue Oct 11 11:47:48 2022] GET /identity/v3/services/object-store => generated 93 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 11 11:47:49.139738 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90419250-051d-47a1-87c3-1e28c7d904a3 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:49.200097 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8028ca-a730-4fe4-b72a-27f0727aa408 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:49.242987 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:49.246434 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60df0aa-3b72-4995-8d2c-5fd9894b49f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:49.247395 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60df0aa-3b72-4995-8d2c-5fd9894b49f0 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:49.247644 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60df0aa-3b72-4995-8d2c-5fd9894b49f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:49.248443 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60df0aa-3b72-4995-8d2c-5fd9894b49f0 None admin] PATH_INFO: `/v3/services/image` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:49.249195 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60df0aa-3b72-4995-8d2c-5fd9894b49f0 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:49.251054 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60df0aa-3b72-4995-8d2c-5fd9894b49f0 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:49.264354 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application [None req-c60df0aa-3b72-4995-8d2c-5fd9894b49f0 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 11 11:47:49.264354 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:47:49.264354 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 11 11:47:49.264354 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 11 11:47:49.264354 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 11 11:47:49.264354 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 11 11:47:49.264354 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 11 11:47:49.264354 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 11:47:49.264354 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 11 11:47:49.264354 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:47:49.264354 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 11 11:47:49.264354 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:47:49.264354 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:47:49.264354 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:47:49.264354 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:47:49.264354 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:47:49.264354 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:47:49.266301 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:47:49.266301 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:47:49.266301 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:47:49.266301 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:47:49.266301 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:47:49.266301 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:47:49.266301 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 11 11:47:49.266301 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 11 11:47:49.266301 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 11 11:47:49.266301 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 11 11:47:49.266301 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:47:49.266301 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:47:49.266301 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:47:49.266301 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:47:49.266301 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:47:49.266301 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:47:49.267986 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:47:49.267986 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application with Lock( Oct 11 11:47:49.267986 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:47:49.267986 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:47:49.267986 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:47:49.267986 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:47:49.267986 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:47:49.267986 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:47:49.267986 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:47:49.267986 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:47:49.267986 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 11 11:47:49.267986 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 11:47:49.267986 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 11 11:47:49.267986 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:47:49.271634 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 369/745] 10.222.0.59 () {56 vars in 1105 bytes} [Tue Oct 11 11:47:49 2022] GET /identity/v3/services/image => generated 86 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 11 11:47:49.271634 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60df0aa-3b72-4995-8d2c-5fd9894b49f0 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:49.309919 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:49.333812 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90419250-051d-47a1-87c3-1e28c7d904a3 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:49.397758 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0868dc40-d628-41b1-8a94-5febb69c262b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:49.398281 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0868dc40-d628-41b1-8a94-5febb69c262b None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:49.398627 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0868dc40-d628-41b1-8a94-5febb69c262b None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:49.398889 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0868dc40-d628-41b1-8a94-5febb69c262b None admin] PATH_INFO: `/v3/services` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:49.399499 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0868dc40-d628-41b1-8a94-5febb69c262b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:49.399794 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0868dc40-d628-41b1-8a94-5febb69c262b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:49.401233 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0868dc40-d628-41b1-8a94-5febb69c262b None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:49.407009 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 377/746] 10.222.0.59 () {56 vars in 1122 bytes} [Tue Oct 11 11:47:49 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:47:49.407699 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0868dc40-d628-41b1-8a94-5febb69c262b None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:49.421613 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60df0aa-3b72-4995-8d2c-5fd9894b49f0 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:49.486057 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed18ab0-6515-4e24-bdd5-11fa64982cc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:49.487154 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed18ab0-6515-4e24-bdd5-11fa64982cc2 None admin] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:49.487644 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed18ab0-6515-4e24-bdd5-11fa64982cc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:49.488129 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed18ab0-6515-4e24-bdd5-11fa64982cc2 None admin] PATH_INFO: `/v3/services` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:49.489741 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed18ab0-6515-4e24-bdd5-11fa64982cc2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:49.492480 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed18ab0-6515-4e24-bdd5-11fa64982cc2 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:49.515621 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:49.522633 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 370/747] 10.222.0.59 () {60 vars in 1141 bytes} [Tue Oct 11 11:47:49 2022] POST /identity/v3/services => generated 242 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 11:47:49.522633 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed18ab0-6515-4e24-bdd5-11fa64982cc2 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:49.544668 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0868dc40-d628-41b1-8a94-5febb69c262b None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:49.601000 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4e0737-e3c1-4ca7-be25-6abaeaef4a1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:49.602544 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4e0737-e3c1-4ca7-be25-6abaeaef4a1d None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:49.603041 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4e0737-e3c1-4ca7-be25-6abaeaef4a1d None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:49.603684 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4e0737-e3c1-4ca7-be25-6abaeaef4a1d None admin] PATH_INFO: `/v3/services` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:49.604656 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4e0737-e3c1-4ca7-be25-6abaeaef4a1d None admin] RBAC: Adding query filter params (name=image) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:49.605124 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4e0737-e3c1-4ca7-be25-6abaeaef4a1d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:49.607353 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4e0737-e3c1-4ca7-be25-6abaeaef4a1d None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:49.621519 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 378/748] 10.222.0.59 () {56 vars in 1108 bytes} [Tue Oct 11 11:47:49 2022] GET /identity/v3/services?name=image => generated 122 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:47:49.623103 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4e0737-e3c1-4ca7-be25-6abaeaef4a1d None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:49.695271 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:49.705996 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed18ab0-6515-4e24-bdd5-11fa64982cc2 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:49.769338 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaf29ca-e30c-42bf-a67f-d0ad10ff657e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:49.770306 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaf29ca-e30c-42bf-a67f-d0ad10ff657e None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:49.770827 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaf29ca-e30c-42bf-a67f-d0ad10ff657e None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:49.771583 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaf29ca-e30c-42bf-a67f-d0ad10ff657e None admin] PATH_INFO: `/v3/services` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:49.772446 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baaf29ca-e30c-42bf-a67f-d0ad10ff657e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:49.773622 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baaf29ca-e30c-42bf-a67f-d0ad10ff657e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:49.775701 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baaf29ca-e30c-42bf-a67f-d0ad10ff657e None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:49.789230 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 371/749] 10.222.0.59 () {56 vars in 1122 bytes} [Tue Oct 11 11:47:49 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:47:49.792123 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaf29ca-e30c-42bf-a67f-d0ad10ff657e None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:49.795001 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4e0737-e3c1-4ca7-be25-6abaeaef4a1d None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:49.863391 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf1d314-4708-418d-b506-be32b03ec24e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:49.864623 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf1d314-4708-418d-b506-be32b03ec24e None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:49.864768 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf1d314-4708-418d-b506-be32b03ec24e None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:49.865192 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf1d314-4708-418d-b506-be32b03ec24e None admin] PATH_INFO: `/v3/services` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:49.866248 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf1d314-4708-418d-b506-be32b03ec24e None admin] RBAC: Adding query filter params (type=image) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:49.867680 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf1d314-4708-418d-b506-be32b03ec24e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:49.870456 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf1d314-4708-418d-b506-be32b03ec24e None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:49.880576 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 379/750] 10.222.0.59 () {56 vars in 1108 bytes} [Tue Oct 11 11:47:49 2022] GET /identity/v3/services?type=image => generated 350 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:47:49.888046 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf1d314-4708-418d-b506-be32b03ec24e None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:49.910058 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:49.989664 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaf29ca-e30c-42bf-a67f-d0ad10ff657e None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:50.057808 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ccab5d-f2b1-474d-b119-ceada7f43c50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:50.059113 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ccab5d-f2b1-474d-b119-ceada7f43c50 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:50.059617 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ccab5d-f2b1-474d-b119-ceada7f43c50 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:50.060099 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ccab5d-f2b1-474d-b119-ceada7f43c50 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:50.061414 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ccab5d-f2b1-474d-b119-ceada7f43c50 None admin] RBAC: Adding query filter params (interface=admin, service_id=3022788117c44b9682bcfe86a7d31099, region_id=RegionOne) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:50.062078 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ccab5d-f2b1-474d-b119-ceada7f43c50 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:50.064969 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ccab5d-f2b1-474d-b119-ceada7f43c50 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:50.075830 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 372/751] 10.222.0.59 () {56 vars in 1249 bytes} [Tue Oct 11 11:47:49 2022] GET /identity/v3/endpoints?service_id=3022788117c44b9682bcfe86a7d31099&interface=admin®ion_id=RegionOne => generated 193 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:47:50.081554 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ccab5d-f2b1-474d-b119-ceada7f43c50 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:50.109468 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf1d314-4708-418d-b506-be32b03ec24e None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:50.112875 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:50.192869 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb30c77-5b0b-4d6d-9a28-5b17fb398264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:50.193931 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb30c77-5b0b-4d6d-9a28-5b17fb398264 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:50.194801 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb30c77-5b0b-4d6d-9a28-5b17fb398264 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:50.196266 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb30c77-5b0b-4d6d-9a28-5b17fb398264 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:50.196996 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb30c77-5b0b-4d6d-9a28-5b17fb398264 None admin] RBAC: Adding query filter params (interface=public, service_id=8b7fb62e0a774c82ae1bc05940165219, region_id=RegionOne) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:50.197305 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb30c77-5b0b-4d6d-9a28-5b17fb398264 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:50.198754 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb30c77-5b0b-4d6d-9a28-5b17fb398264 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:50.210213 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 380/752] 10.222.0.59 () {56 vars in 1251 bytes} [Tue Oct 11 11:47:49 2022] GET /identity/v3/endpoints?service_id=8b7fb62e0a774c82ae1bc05940165219&interface=public®ion_id=RegionOne => generated 194 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:47:50.211205 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb30c77-5b0b-4d6d-9a28-5b17fb398264 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:50.221531 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ccab5d-f2b1-474d-b119-ceada7f43c50 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:50.292082 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05501e1c-6888-4e68-8f46-511a6dcad655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:50.292462 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:50.293319 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05501e1c-6888-4e68-8f46-511a6dcad655 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:50.293873 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05501e1c-6888-4e68-8f46-511a6dcad655 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:50.294436 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05501e1c-6888-4e68-8f46-511a6dcad655 None admin] PATH_INFO: `/v3/services` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:50.295672 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05501e1c-6888-4e68-8f46-511a6dcad655 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:50.298304 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05501e1c-6888-4e68-8f46-511a6dcad655 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:50.318591 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 373/753] 10.222.0.59 () {56 vars in 1087 bytes} [Tue Oct 11 11:47:50 2022] GET /identity/v3/services => generated 1957 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 11 11:47:50.356777 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb30c77-5b0b-4d6d-9a28-5b17fb398264 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:50.424157 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2024a94b-3aef-4016-8e45-8cc1b0cab6d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:50.425205 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2024a94b-3aef-4016-8e45-8cc1b0cab6d1 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:50.425692 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2024a94b-3aef-4016-8e45-8cc1b0cab6d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:50.426164 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2024a94b-3aef-4016-8e45-8cc1b0cab6d1 None admin] PATH_INFO: `/v3/services` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:50.427331 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2024a94b-3aef-4016-8e45-8cc1b0cab6d1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:50.429988 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2024a94b-3aef-4016-8e45-8cc1b0cab6d1 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:50.444277 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 381/754] 10.222.0.59 () {56 vars in 1087 bytes} [Tue Oct 11 11:47:50 2022] GET /identity/v3/services => generated 1957 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 11:47:51.023271 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b2228a-1385-47db-9ae1-097a5433797a None None] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:51.023790 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b2228a-1385-47db-9ae1-097a5433797a None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:51.023790 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b2228a-1385-47db-9ae1-097a5433797a None None] PATH_INFO: `/` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:51.025365 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 374/755] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:47:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:47:51.030976 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f685d39d-16fe-464d-93ed-0ef7c6e12d92 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:51.031265 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f685d39d-16fe-464d-93ed-0ef7c6e12d92 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:51.031541 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f685d39d-16fe-464d-93ed-0ef7c6e12d92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:51.050580 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-f685d39d-16fe-464d-93ed-0ef7c6e12d92 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:47:51.051762 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-f685d39d-16fe-464d-93ed-0ef7c6e12d92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:47:51.128178 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 382/756] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:47:51 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 11 11:47:51.392211 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b2228a-1385-47db-9ae1-097a5433797a None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:51.483859 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:51.506424 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b2228a-1385-47db-9ae1-097a5433797a None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:51.542704 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d626c8-0537-4e03-8955-024702245032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:51.543610 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d626c8-0537-4e03-8955-024702245032 None admin] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:51.544029 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d626c8-0537-4e03-8955-024702245032 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:51.544438 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d626c8-0537-4e03-8955-024702245032 None admin] PATH_INFO: `/v3/users` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:51.545245 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d626c8-0537-4e03-8955-024702245032 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:51.546835 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d626c8-0537-4e03-8955-024702245032 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:51.565240 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.sql.core [None req-03d626c8-0537-4e03-8955-024702245032 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 11 11:47:51.565240 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 11 11:47:51.565240 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [parameters: {'user_id': '67f87645b9ad409881abfd199b2d4f0e', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 11 11:47:51.565240 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81433) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 11 11:47:51.573328 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application [None req-03d626c8-0537-4e03-8955-024702245032 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 11 11:47:51.573328 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:47:51.573328 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 11 11:47:51.573328 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 11 11:47:51.573328 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 11 11:47:51.573328 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 11 11:47:51.573328 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 11 11:47:51.573328 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application result = self._query(query) Oct 11 11:47:51.573328 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 11 11:47:51.573328 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application conn.query(q) Oct 11 11:47:51.573328 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 11 11:47:51.573328 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 11 11:47:51.573328 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 11 11:47:51.573328 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application result.read() Oct 11 11:47:51.573328 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 11 11:47:51.574414 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 11 11:47:51.574414 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 11 11:47:51.574414 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 11 11:47:51.574414 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 11 11:47:51.574414 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 11 11:47:51.574414 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 11 11:47:51.574414 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 11 11:47:51.574414 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 11 11:47:51.574414 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:47:51.574414 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 11 11:47:51.574414 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:47:51.574414 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:47:51.574414 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 11 11:47:51.574414 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 11 11:47:51.574414 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Oct 11 11:47:51.574414 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Oct 11 11:47:51.574414 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 11 11:47:51.574414 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application next(self.gen) Oct 11 11:47:51.575643 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 11 11:47:51.575643 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application yield resource Oct 11 11:47:51.575643 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 11 11:47:51.575643 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application next(self.gen) Oct 11 11:47:51.575643 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 11 11:47:51.575643 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application self.session.rollback() Oct 11 11:47:51.575643 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 11 11:47:51.575643 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application self.force_reraise() Oct 11 11:47:51.575643 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 11 11:47:51.575643 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application raise self.value Oct 11 11:47:51.575643 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 11 11:47:51.575643 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 11 11:47:51.575643 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 11 11:47:51.575643 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application session.commit() Oct 11 11:47:51.575643 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 11 11:47:51.575643 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 11 11:47:51.575643 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 11 11:47:51.576679 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application self._prepare_impl() Oct 11 11:47:51.576679 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 11 11:47:51.576679 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application self.session.flush() Oct 11 11:47:51.576679 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 11 11:47:51.576679 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application self._flush(objects) Oct 11 11:47:51.576679 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 11 11:47:51.576679 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 11 11:47:51.576679 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 11 11:47:51.576679 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application compat.raise_( Oct 11 11:47:51.576679 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 11 11:47:51.576679 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application raise exception Oct 11 11:47:51.576679 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 11 11:47:51.576679 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application flush_context.execute() Oct 11 11:47:51.576679 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 11 11:47:51.576679 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application rec.execute(self) Oct 11 11:47:51.576679 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 11 11:47:51.576679 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 11 11:47:51.577530 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 11 11:47:51.577530 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 11 11:47:51.577530 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Oct 11 11:47:51.577530 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application result = connection._execute_20( Oct 11 11:47:51.577530 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 11 11:47:51.577530 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 11 11:47:51.577530 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 11 11:47:51.577530 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 11 11:47:51.577530 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 11 11:47:51.577530 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 11 11:47:51.577530 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 11 11:47:51.577530 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 11 11:47:51.577530 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 11 11:47:51.577530 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 11 11:47:51.577530 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 11 11:47:51.577530 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application raise exception Oct 11 11:47:51.578370 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 11 11:47:51.578370 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 11 11:47:51.578370 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 11 11:47:51.578370 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 11 11:47:51.578370 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 11 11:47:51.578370 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application result = self._query(query) Oct 11 11:47:51.578370 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 11 11:47:51.578370 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application conn.query(q) Oct 11 11:47:51.578370 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 11 11:47:51.578370 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 11 11:47:51.578370 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 11 11:47:51.578370 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application result.read() Oct 11 11:47:51.578370 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 11 11:47:51.578370 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 11 11:47:51.578370 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 11 11:47:51.578370 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 11 11:47:51.579463 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 11 11:47:51.579463 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 11 11:47:51.579463 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 11 11:47:51.579463 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 11 11:47:51.579463 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 11 11:47:51.579463 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 11 11:47:51.579463 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application [parameters: {'user_id': '67f87645b9ad409881abfd199b2d4f0e', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 11 11:47:51.579463 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 11 11:47:51.579463 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:47:51.579463 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 11 11:47:51.579463 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:47:51.579463 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:47:51.579463 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:47:51.579463 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:47:51.579463 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:47:51.580504 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:47:51.580504 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:47:51.580504 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:47:51.580504 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:47:51.580504 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:47:51.580504 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:47:51.580504 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:47:51.580504 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Oct 11 11:47:51.580504 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Oct 11 11:47:51.580504 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:47:51.580504 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:47:51.580504 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 11 11:47:51.580504 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 11:47:51.580504 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 11 11:47:51.580504 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 11:47:51.580504 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Oct 11 11:47:51.580504 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Oct 11 11:47:51.581507 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Oct 11 11:47:51.581507 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Oct 11 11:47:51.581507 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 11 11:47:51.581507 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 11 11:47:51.581507 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 11 11:47:51.581507 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:47:51.583637 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 375/757] 10.222.0.59 () {60 vars in 1131 bytes} [Tue Oct 11 11:47:51 2022] POST /identity/v3/users => generated 160 bytes in 192 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:47:51.589836 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f685d39d-16fe-464d-93ed-0ef7c6e12d92 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:51.667279 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:51.705404 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f685d39d-16fe-464d-93ed-0ef7c6e12d92 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:51.744935 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def0bdb6-32e1-4c81-a799-f97e90e54bcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:51.745877 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def0bdb6-32e1-4c81-a799-f97e90e54bcd None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:51.746345 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def0bdb6-32e1-4c81-a799-f97e90e54bcd None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:51.746801 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def0bdb6-32e1-4c81-a799-f97e90e54bcd None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:51.755390 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def0bdb6-32e1-4c81-a799-f97e90e54bcd None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:51.757266 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def0bdb6-32e1-4c81-a799-f97e90e54bcd None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:51.770549 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application [None req-def0bdb6-32e1-4c81-a799-f97e90e54bcd None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 11 11:47:51.770549 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:47:51.770549 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:47:51.770549 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:47:51.770549 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:47:51.770549 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:47:51.770549 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:47:51.770549 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:47:51.770549 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:47:51.770549 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:47:51.770549 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:47:51.770549 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:47:51.770549 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 11 11:47:51.770549 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 11 11:47:51.770549 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 11 11:47:51.770549 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 11 11:47:51.771684 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:47:51.771684 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:47:51.771684 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 11 11:47:51.771684 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 11:47:51.771684 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 11 11:47:51.771684 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 11:47:51.771684 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:47:51.771684 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:47:51.771684 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:47:51.771684 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:47:51.771684 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:47:51.771684 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application with Lock( Oct 11 11:47:51.771684 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:47:51.771684 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:47:51.771684 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:47:51.771684 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:47:51.771684 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:47:51.771684 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:47:51.772640 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:47:51.772640 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:47:51.772640 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 11 11:47:51.772640 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 11 11:47:51.772640 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 11 11:47:51.772640 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 11 11:47:51.772640 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 11 11:47:51.772640 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 11 11:47:51.772640 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 11 11:47:51.772640 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:47:51.772640 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 383/758] 10.222.0.59 () {56 vars in 1099 bytes} [Tue Oct 11 11:47:51 2022] GET /identity/v3/users/cinder => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:47:51.780121 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d626c8-0537-4e03-8955-024702245032 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:51.957417 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d626c8-0537-4e03-8955-024702245032 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:52.000484 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440f8619-3795-48b2-8d00-e9ba18fcb4f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:52.001145 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440f8619-3795-48b2-8d00-e9ba18fcb4f2 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:52.001420 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440f8619-3795-48b2-8d00-e9ba18fcb4f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:52.001697 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440f8619-3795-48b2-8d00-e9ba18fcb4f2 None admin] PATH_INFO: `/v3/users` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:52.002428 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440f8619-3795-48b2-8d00-e9ba18fcb4f2 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:52.002755 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440f8619-3795-48b2-8d00-e9ba18fcb4f2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:52.004523 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440f8619-3795-48b2-8d00-e9ba18fcb4f2 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:52.075030 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 376/759] 10.222.0.59 () {56 vars in 1101 bytes} [Tue Oct 11 11:47:51 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:47:52.254205 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1805d6c-34ec-4cd8-a505-fda5a2c85bc1 None None] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:52.254769 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1805d6c-34ec-4cd8-a505-fda5a2c85bc1 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:52.255332 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1805d6c-34ec-4cd8-a505-fda5a2c85bc1 None None] PATH_INFO: `/` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:52.258334 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 384/760] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:47:52 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:47:52.275859 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9283e716-7b99-48d4-98d4-d560a2bc26fe None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:52.276157 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9283e716-7b99-48d4-98d4-d560a2bc26fe None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:52.276411 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9283e716-7b99-48d4-98d4-d560a2bc26fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:52.290132 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:52.312169 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-9283e716-7b99-48d4-98d4-d560a2bc26fe None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:47:52.313941 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-9283e716-7b99-48d4-98d4-d560a2bc26fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:47:52.430713 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 377/761] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:47:52 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 11:47:52.693474 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1805d6c-34ec-4cd8-a505-fda5a2c85bc1 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:52.826631 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1805d6c-34ec-4cd8-a505-fda5a2c85bc1 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:52.861605 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:52.872429 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7131c33-f317-4523-a559-ded42d9fe41e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:52.873081 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7131c33-f317-4523-a559-ded42d9fe41e None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:52.873354 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7131c33-f317-4523-a559-ded42d9fe41e None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:52.873622 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7131c33-f317-4523-a559-ded42d9fe41e None admin] PATH_INFO: `/v3/services/compute` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:52.874210 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7131c33-f317-4523-a559-ded42d9fe41e None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:52.875801 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7131c33-f317-4523-a559-ded42d9fe41e None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:52.885578 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application [None req-b7131c33-f317-4523-a559-ded42d9fe41e None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 11 11:47:52.885578 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:47:52.885578 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 11 11:47:52.885578 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 11 11:47:52.885578 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 11 11:47:52.885578 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 11 11:47:52.885578 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 11 11:47:52.885578 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 11:47:52.885578 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 11 11:47:52.885578 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:47:52.885578 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 11 11:47:52.885578 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:47:52.885578 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:47:52.885578 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:47:52.885578 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:47:52.885578 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:47:52.885578 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:47:52.887552 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:47:52.887552 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:47:52.887552 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:47:52.887552 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:47:52.887552 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:47:52.887552 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:47:52.887552 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 11 11:47:52.887552 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 11 11:47:52.887552 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 11 11:47:52.887552 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 11 11:47:52.887552 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:47:52.887552 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:47:52.887552 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:47:52.887552 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:47:52.887552 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:47:52.887552 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:47:52.889123 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:47:52.889123 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application with Lock( Oct 11 11:47:52.889123 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:47:52.889123 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:47:52.889123 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:47:52.889123 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:47:52.889123 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:47:52.889123 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:47:52.889123 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:47:52.889123 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:47:52.889123 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 11 11:47:52.889123 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 11:47:52.889123 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 11 11:47:52.889123 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:47:52.891796 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 385/762] 10.222.0.59 () {56 vars in 1111 bytes} [Tue Oct 11 11:47:52 2022] GET /identity/v3/services/compute => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 11 11:47:52.893223 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9283e716-7b99-48d4-98d4-d560a2bc26fe None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:53.037825 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9283e716-7b99-48d4-98d4-d560a2bc26fe None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:53.088860 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97508e28-ddc4-4804-bfc7-e4a9f3197414 None None] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:53.089151 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97508e28-ddc4-4804-bfc7-e4a9f3197414 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:53.089421 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97508e28-ddc4-4804-bfc7-e4a9f3197414 None None] PATH_INFO: `/` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:53.091099 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 386/763] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:47:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 11 11:47:53.097023 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34fdec2-12f2-4b7a-a7cd-35800941363b None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:53.097323 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34fdec2-12f2-4b7a-a7cd-35800941363b None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:53.097572 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34fdec2-12f2-4b7a-a7cd-35800941363b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:53.104680 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc73e95-2e6e-4ecd-b46d-e234dbb80b56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:53.106880 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc73e95-2e6e-4ecd-b46d-e234dbb80b56 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:53.107168 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc73e95-2e6e-4ecd-b46d-e234dbb80b56 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:53.107437 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc73e95-2e6e-4ecd-b46d-e234dbb80b56 None admin] PATH_INFO: `/v3/services` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:53.108097 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc73e95-2e6e-4ecd-b46d-e234dbb80b56 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:53.108395 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc73e95-2e6e-4ecd-b46d-e234dbb80b56 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:53.109833 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc73e95-2e6e-4ecd-b46d-e234dbb80b56 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:53.111635 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:53.124470 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 378/764] 10.222.0.59 () {56 vars in 1112 bytes} [Tue Oct 11 11:47:52 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:47:53.126099 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc73e95-2e6e-4ecd-b46d-e234dbb80b56 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:53.128111 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-a34fdec2-12f2-4b7a-a7cd-35800941363b None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:47:53.130154 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-a34fdec2-12f2-4b7a-a7cd-35800941363b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:47:53.230606 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 387/765] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:47:53 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 11:47:53.276761 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc73e95-2e6e-4ecd-b46d-e234dbb80b56 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:53.331311 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1486c2-c7cb-40a1-8723-a70106f05d8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:53.332140 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1486c2-c7cb-40a1-8723-a70106f05d8a None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:53.332513 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1486c2-c7cb-40a1-8723-a70106f05d8a None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:53.332873 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1486c2-c7cb-40a1-8723-a70106f05d8a None admin] PATH_INFO: `/v3/services` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:53.333767 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1486c2-c7cb-40a1-8723-a70106f05d8a None admin] RBAC: Adding query filter params (type=compute) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:53.334183 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1486c2-c7cb-40a1-8723-a70106f05d8a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:53.336133 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1486c2-c7cb-40a1-8723-a70106f05d8a None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:53.350603 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 379/766] 10.222.0.59 () {56 vars in 1112 bytes} [Tue Oct 11 11:47:53 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 11:47:53.354512 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34fdec2-12f2-4b7a-a7cd-35800941363b None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:53.365286 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03405da-f52f-4219-8ab0-da61d667418f None None] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:53.365638 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03405da-f52f-4219-8ab0-da61d667418f None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:53.365830 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03405da-f52f-4219-8ab0-da61d667418f None None] PATH_INFO: `/` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:53.367395 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 380/767] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:47:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 11 11:47:53.372992 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecaef4f-7451-4572-bab3-0d3f447fedf1 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:53.373273 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecaef4f-7451-4572-bab3-0d3f447fedf1 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:53.374000 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecaef4f-7451-4572-bab3-0d3f447fedf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:53.392685 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-5ecaef4f-7451-4572-bab3-0d3f447fedf1 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:47:53.393883 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-5ecaef4f-7451-4572-bab3-0d3f447fedf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:47:53.468326 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:53.489604 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 381/768] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:47:53 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 11 11:47:53.506339 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34fdec2-12f2-4b7a-a7cd-35800941363b None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:53.532281 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecaef4f-7451-4572-bab3-0d3f447fedf1 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:53.533234 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:53.550321 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e3dcfe-4f97-4e60-a3e3-ef1b609ad0be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:53.551007 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e3dcfe-4f97-4e60-a3e3-ef1b609ad0be None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:53.551284 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e3dcfe-4f97-4e60-a3e3-ef1b609ad0be None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:53.551552 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e3dcfe-4f97-4e60-a3e3-ef1b609ad0be None admin] PATH_INFO: `/v3/endpoints` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:53.552582 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e3dcfe-4f97-4e60-a3e3-ef1b609ad0be None admin] RBAC: Adding query filter params (interface=public, service_id=1c2ab1c111824e7ca549be2d95f1c4a7, region_id=RegionOne) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:53.552582 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e3dcfe-4f97-4e60-a3e3-ef1b609ad0be None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:53.554146 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e3dcfe-4f97-4e60-a3e3-ef1b609ad0be None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:53.560281 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 388/769] 10.222.0.59 () {56 vars in 1251 bytes} [Tue Oct 11 11:47:53 2022] GET /identity/v3/endpoints?service_id=1c2ab1c111824e7ca549be2d95f1c4a7&interface=public®ion_id=RegionOne => generated 194 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:47:53.566291 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e3dcfe-4f97-4e60-a3e3-ef1b609ad0be None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:53.681898 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecaef4f-7451-4572-bab3-0d3f447fedf1 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:53.692793 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e3dcfe-4f97-4e60-a3e3-ef1b609ad0be None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:53.735956 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192c7365-fe1b-42fe-8867-47391e92636a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:53.736744 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192c7365-fe1b-42fe-8867-47391e92636a None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:53.737023 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192c7365-fe1b-42fe-8867-47391e92636a None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:53.737319 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192c7365-fe1b-42fe-8867-47391e92636a None admin] PATH_INFO: `/v3/services/image` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:53.737925 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192c7365-fe1b-42fe-8867-47391e92636a None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:53.738207 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27375f4c-c060-4773-8e3c-119954e86b73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:53.738941 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27375f4c-c060-4773-8e3c-119954e86b73 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:53.739198 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27375f4c-c060-4773-8e3c-119954e86b73 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:53.739339 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192c7365-fe1b-42fe-8867-47391e92636a None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:53.739467 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27375f4c-c060-4773-8e3c-119954e86b73 None admin] PATH_INFO: `/v3/services` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:53.740253 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27375f4c-c060-4773-8e3c-119954e86b73 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:53.741898 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27375f4c-c060-4773-8e3c-119954e86b73 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:53.749328 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application [None req-192c7365-fe1b-42fe-8867-47391e92636a None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 11 11:47:53.749328 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:47:53.749328 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 11 11:47:53.749328 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 11 11:47:53.749328 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 11 11:47:53.749328 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 11 11:47:53.749328 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 11 11:47:53.749328 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 11:47:53.749328 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 11 11:47:53.749328 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:47:53.749328 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 11 11:47:53.749328 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:47:53.749328 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:47:53.749328 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:47:53.749328 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:47:53.749328 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:47:53.749328 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:47:53.751360 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:47:53.751360 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:47:53.751360 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:47:53.751360 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:47:53.751360 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:47:53.751360 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:47:53.751360 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 11 11:47:53.751360 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 11 11:47:53.751360 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 11 11:47:53.751360 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 11 11:47:53.751360 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:47:53.751360 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:47:53.751360 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:47:53.751360 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:47:53.751360 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:47:53.751360 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:47:53.752648 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:47:53.752648 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application with Lock( Oct 11 11:47:53.752648 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:47:53.752648 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:47:53.752648 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:47:53.752648 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:47:53.752648 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:47:53.752648 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:47:53.752648 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:47:53.752648 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:47:53.752648 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 11 11:47:53.752648 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 11:47:53.752648 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 11 11:47:53.752648 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:47:53.754119 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 389/770] 10.222.0.59 () {56 vars in 1087 bytes} [Tue Oct 11 11:47:53 2022] GET /identity/v3/services => generated 1957 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 11 11:47:53.754264 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 382/771] 10.222.0.59 () {56 vars in 1105 bytes} [Tue Oct 11 11:47:53 2022] GET /identity/v3/services/image => generated 86 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 11 11:47:53.764168 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27375f4c-c060-4773-8e3c-119954e86b73 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:53.834183 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192c7365-fe1b-42fe-8867-47391e92636a None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:53.855659 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:53.932716 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:53.960391 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27375f4c-c060-4773-8e3c-119954e86b73 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:54.000962 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192c7365-fe1b-42fe-8867-47391e92636a None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:54.026161 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde06570-e001-4b59-b64b-c0258fe2bbb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:54.027398 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde06570-e001-4b59-b64b-c0258fe2bbb5 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:54.027896 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde06570-e001-4b59-b64b-c0258fe2bbb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:54.028388 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde06570-e001-4b59-b64b-c0258fe2bbb5 None admin] PATH_INFO: `/v3/services` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:54.029814 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde06570-e001-4b59-b64b-c0258fe2bbb5 None admin] RBAC: Adding query filter params (name=image) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:54.030508 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde06570-e001-4b59-b64b-c0258fe2bbb5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:54.033419 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde06570-e001-4b59-b64b-c0258fe2bbb5 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:54.038678 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254cdd40-696e-4d21-8b05-007c375a7847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:54.039319 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254cdd40-696e-4d21-8b05-007c375a7847 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:54.039600 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254cdd40-696e-4d21-8b05-007c375a7847 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:54.039872 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254cdd40-696e-4d21-8b05-007c375a7847 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:54.040467 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254cdd40-696e-4d21-8b05-007c375a7847 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:54.041861 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254cdd40-696e-4d21-8b05-007c375a7847 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:54.049032 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 390/772] 10.222.0.59 () {56 vars in 1108 bytes} [Tue Oct 11 11:47:53 2022] GET /identity/v3/services?name=image => generated 122 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:47:54.050752 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde06570-e001-4b59-b64b-c0258fe2bbb5 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:54.060898 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application [None req-254cdd40-696e-4d21-8b05-007c375a7847 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 11 11:47:54.060898 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:47:54.060898 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 11 11:47:54.060898 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 11 11:47:54.060898 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 11 11:47:54.060898 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 11 11:47:54.060898 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 11 11:47:54.060898 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 11:47:54.060898 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 11 11:47:54.060898 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:47:54.060898 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 11 11:47:54.060898 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:47:54.060898 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:47:54.060898 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:47:54.060898 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:47:54.060898 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:47:54.060898 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:47:54.062184 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:47:54.062184 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:47:54.062184 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:47:54.062184 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:47:54.062184 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:47:54.062184 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:47:54.062184 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 11 11:47:54.062184 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 11 11:47:54.062184 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 11 11:47:54.062184 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 11 11:47:54.062184 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:47:54.062184 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:47:54.062184 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:47:54.062184 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:47:54.062184 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:47:54.062184 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:47:54.063252 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:47:54.063252 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application with Lock( Oct 11 11:47:54.063252 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:47:54.063252 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:47:54.063252 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:47:54.063252 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:47:54.063252 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:47:54.063252 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:47:54.063252 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:47:54.063252 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:47:54.063252 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 11 11:47:54.063252 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 11:47:54.063252 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 11 11:47:54.063252 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:47:54.064007 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 383/773] 10.222.0.59 () {56 vars in 1126 bytes} [Tue Oct 11 11:47:53 2022] GET /identity/v3/services/object-store => generated 93 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:47:54.070357 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254cdd40-696e-4d21-8b05-007c375a7847 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:54.204329 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254cdd40-696e-4d21-8b05-007c375a7847 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:54.230722 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:54.250210 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2080cc89-8b4b-42bd-b15e-4139fe63a288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:54.251137 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2080cc89-8b4b-42bd-b15e-4139fe63a288 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:54.251407 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2080cc89-8b4b-42bd-b15e-4139fe63a288 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:54.251686 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2080cc89-8b4b-42bd-b15e-4139fe63a288 None admin] PATH_INFO: `/v3/services` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:54.252351 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2080cc89-8b4b-42bd-b15e-4139fe63a288 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:54.252655 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2080cc89-8b4b-42bd-b15e-4139fe63a288 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:54.254111 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2080cc89-8b4b-42bd-b15e-4139fe63a288 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:54.265183 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 384/774] 10.222.0.59 () {56 vars in 1122 bytes} [Tue Oct 11 11:47:54 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 11:47:54.266956 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2080cc89-8b4b-42bd-b15e-4139fe63a288 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:54.334810 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde06570-e001-4b59-b64b-c0258fe2bbb5 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:54.358389 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:54.396917 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f243f31-9b34-49e4-8fad-0ccf3f2eceb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:54.397933 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f243f31-9b34-49e4-8fad-0ccf3f2eceb6 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:54.398383 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f243f31-9b34-49e4-8fad-0ccf3f2eceb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:54.398850 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f243f31-9b34-49e4-8fad-0ccf3f2eceb6 None admin] PATH_INFO: `/v3/services` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:54.399899 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f243f31-9b34-49e4-8fad-0ccf3f2eceb6 None admin] RBAC: Adding query filter params (type=image) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:54.400394 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f243f31-9b34-49e4-8fad-0ccf3f2eceb6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:54.402873 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f243f31-9b34-49e4-8fad-0ccf3f2eceb6 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:54.412109 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 391/775] 10.222.0.59 () {56 vars in 1108 bytes} [Tue Oct 11 11:47:54 2022] GET /identity/v3/services?type=image => generated 350 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:47:54.419335 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f243f31-9b34-49e4-8fad-0ccf3f2eceb6 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:54.425976 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2080cc89-8b4b-42bd-b15e-4139fe63a288 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:54.480911 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8427f5-f179-4a6a-9963-ceb5e1a1f4c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:54.481915 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8427f5-f179-4a6a-9963-ceb5e1a1f4c5 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:54.482364 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8427f5-f179-4a6a-9963-ceb5e1a1f4c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:54.482840 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8427f5-f179-4a6a-9963-ceb5e1a1f4c5 None admin] PATH_INFO: `/v3/services` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:54.484021 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8427f5-f179-4a6a-9963-ceb5e1a1f4c5 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:54.484550 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8427f5-f179-4a6a-9963-ceb5e1a1f4c5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:54.487042 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8427f5-f179-4a6a-9963-ceb5e1a1f4c5 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:54.502727 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 385/776] 10.222.0.59 () {56 vars in 1122 bytes} [Tue Oct 11 11:47:54 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:47:54.505009 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8427f5-f179-4a6a-9963-ceb5e1a1f4c5 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:54.569023 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:54.589221 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f243f31-9b34-49e4-8fad-0ccf3f2eceb6 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:54.635589 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25705e54-0a4e-4490-819e-67a63721b245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:54.636352 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25705e54-0a4e-4490-819e-67a63721b245 None admin] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:54.636687 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25705e54-0a4e-4490-819e-67a63721b245 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:54.637016 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25705e54-0a4e-4490-819e-67a63721b245 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:54.637311 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8427f5-f179-4a6a-9963-ceb5e1a1f4c5 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:54.637847 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25705e54-0a4e-4490-819e-67a63721b245 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:54.639409 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25705e54-0a4e-4490-819e-67a63721b245 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:54.666603 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 392/777] 10.222.0.59 () {60 vars in 1144 bytes} [Tue Oct 11 11:47:54 2022] POST /identity/v3/endpoints => generated 327 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 11:47:54.707895 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27feabca-b88c-43ee-a3b4-59058dd03565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:54.708946 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27feabca-b88c-43ee-a3b4-59058dd03565 None admin] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:54.709510 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27feabca-b88c-43ee-a3b4-59058dd03565 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:54.710076 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27feabca-b88c-43ee-a3b4-59058dd03565 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:54.711203 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27feabca-b88c-43ee-a3b4-59058dd03565 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:54.713072 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27feabca-b88c-43ee-a3b4-59058dd03565 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:54.738057 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 386/778] 10.222.0.59 () {60 vars in 1144 bytes} [Tue Oct 11 11:47:54 2022] POST /identity/v3/endpoints => generated 325 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 11:47:56.209816 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903d6cb6-10ed-4df8-a8a9-df01cafc5038 None None] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:56.210467 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903d6cb6-10ed-4df8-a8a9-df01cafc5038 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:56.210467 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903d6cb6-10ed-4df8-a8a9-df01cafc5038 None None] PATH_INFO: `/` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:56.211950 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 393/779] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:47:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:47:56.217584 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95b0c6c-92c2-48c2-b8a1-e2a54e1f5126 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:56.218069 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95b0c6c-92c2-48c2-b8a1-e2a54e1f5126 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:56.218069 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95b0c6c-92c2-48c2-b8a1-e2a54e1f5126 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:56.239230 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-f95b0c6c-92c2-48c2-b8a1-e2a54e1f5126 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:47:56.240544 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-f95b0c6c-92c2-48c2-b8a1-e2a54e1f5126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:47:56.338633 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 387/780] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:47:56 2022] POST /identity/v3/auth/tokens => generated 1962 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 11:47:56.732024 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903d6cb6-10ed-4df8-a8a9-df01cafc5038 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:56.789925 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:56.887430 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a479cf-1b4d-40a6-b041-b337180a47a5 None None] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:56.887743 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a479cf-1b4d-40a6-b041-b337180a47a5 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:56.888019 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a479cf-1b4d-40a6-b041-b337180a47a5 None None] PATH_INFO: `/` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:56.889426 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 388/781] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:47:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:47:56.897606 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9806fa58-faae-49c0-af02-c7a2e59b7509 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:56.898062 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9806fa58-faae-49c0-af02-c7a2e59b7509 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:56.898526 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9806fa58-faae-49c0-af02-c7a2e59b7509 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:56.925419 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-9806fa58-faae-49c0-af02-c7a2e59b7509 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:47:56.926326 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903d6cb6-10ed-4df8-a8a9-df01cafc5038 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:56.927224 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-9806fa58-faae-49c0-af02-c7a2e59b7509 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:47:56.964631 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:56.984189 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b54d577-a011-4c1d-832d-da98a2e4e3d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:56.984789 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b54d577-a011-4c1d-832d-da98a2e4e3d8 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:56.985062 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b54d577-a011-4c1d-832d-da98a2e4e3d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:56.985329 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b54d577-a011-4c1d-832d-da98a2e4e3d8 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:56.985924 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b54d577-a011-4c1d-832d-da98a2e4e3d8 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:56.987355 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b54d577-a011-4c1d-832d-da98a2e4e3d8 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:56.998875 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application [None req-2b54d577-a011-4c1d-832d-da98a2e4e3d8 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 11 11:47:56.998875 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:47:56.998875 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 11 11:47:56.998875 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 11 11:47:56.998875 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 11 11:47:56.998875 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 11 11:47:56.998875 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 11 11:47:56.998875 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 11:47:56.998875 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 11 11:47:56.998875 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:47:56.998875 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 11 11:47:56.998875 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:47:56.998875 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:47:56.998875 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:47:56.998875 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:47:56.998875 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:47:56.998875 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:47:57.000614 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:47:57.000614 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:47:57.000614 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:47:57.000614 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:47:57.000614 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:47:57.000614 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:47:57.000614 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 11 11:47:57.000614 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 11 11:47:57.000614 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 11 11:47:57.000614 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 11 11:47:57.000614 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:47:57.000614 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:47:57.000614 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:47:57.000614 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:47:57.000614 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:47:57.000614 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:47:57.001558 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:47:57.001558 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application with Lock( Oct 11 11:47:57.001558 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:47:57.001558 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:47:57.001558 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:47:57.001558 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:47:57.001558 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:47:57.001558 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:47:57.001558 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:47:57.001558 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:47:57.001558 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 11 11:47:57.001558 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 11:47:57.001558 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 11 11:47:57.001558 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:47:57.001558 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 394/782] 10.222.0.59 () {56 vars in 1111 bytes} [Tue Oct 11 11:47:56 2022] GET /identity/v3/services/compute => generated 88 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 11 11:47:57.006616 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b54d577-a011-4c1d-832d-da98a2e4e3d8 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:57.025605 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 389/783] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:47:56 2022] POST /identity/v3/auth/tokens => generated 1962 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 11 11:47:57.118690 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:57.128641 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b54d577-a011-4c1d-832d-da98a2e4e3d8 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:57.176016 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1817b0-cc1f-4cbe-9b15-38ce55d5c99d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:57.176844 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1817b0-cc1f-4cbe-9b15-38ce55d5c99d None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:57.177316 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1817b0-cc1f-4cbe-9b15-38ce55d5c99d None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:57.177793 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1817b0-cc1f-4cbe-9b15-38ce55d5c99d None admin] PATH_INFO: `/v3/services` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:57.178995 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1817b0-cc1f-4cbe-9b15-38ce55d5c99d None admin] RBAC: Adding query filter params (name=compute) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:57.179540 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1817b0-cc1f-4cbe-9b15-38ce55d5c99d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:57.182138 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1817b0-cc1f-4cbe-9b15-38ce55d5c99d None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:57.195015 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 395/784] 10.222.0.59 () {56 vars in 1112 bytes} [Tue Oct 11 11:47:57 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:47:57.201351 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9806fa58-faae-49c0-af02-c7a2e59b7509 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:57.289350 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1817b0-cc1f-4cbe-9b15-38ce55d5c99d None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:57.344448 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9806fa58-faae-49c0-af02-c7a2e59b7509 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:57.387928 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5851f72e-1fb8-44c5-a027-24f7d7b87baa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:57.388756 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5851f72e-1fb8-44c5-a027-24f7d7b87baa None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:57.389186 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5851f72e-1fb8-44c5-a027-24f7d7b87baa None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:57.389340 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5851f72e-1fb8-44c5-a027-24f7d7b87baa None admin] PATH_INFO: `/v3/services` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:57.389943 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5851f72e-1fb8-44c5-a027-24f7d7b87baa None admin] RBAC: Adding query filter params (type=compute) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:57.390254 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5851f72e-1fb8-44c5-a027-24f7d7b87baa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:57.392323 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5851f72e-1fb8-44c5-a027-24f7d7b87baa None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:57.406199 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 390/785] 10.222.0.59 () {56 vars in 1112 bytes} [Tue Oct 11 11:47:57 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 11:47:57.408343 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5851f72e-1fb8-44c5-a027-24f7d7b87baa None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:57.439815 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:57.440777 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1817b0-cc1f-4cbe-9b15-38ce55d5c99d None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:57.486244 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f496f1ba-e90c-43fe-8876-dbfb66100613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:57.486912 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f496f1ba-e90c-43fe-8876-dbfb66100613 None admin] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:57.487185 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f496f1ba-e90c-43fe-8876-dbfb66100613 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:57.487457 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f496f1ba-e90c-43fe-8876-dbfb66100613 None admin] PATH_INFO: `/v3/domains` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:57.488124 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f496f1ba-e90c-43fe-8876-dbfb66100613 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:57.489582 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f496f1ba-e90c-43fe-8876-dbfb66100613 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:57.499095 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.sql.core [None req-f496f1ba-e90c-43fe-8876-dbfb66100613 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 11 11:47:57.499095 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 11 11:47:57.499095 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [parameters: {'id': 'cf52aaa93edf45f3b25fb5165173951a', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 11 11:47:57.499095 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81432) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 11 11:47:57.517369 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application [None req-f496f1ba-e90c-43fe-8876-dbfb66100613 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 11 11:47:57.517369 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:47:57.517369 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 11 11:47:57.517369 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 11 11:47:57.517369 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 11 11:47:57.517369 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 11 11:47:57.517369 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 11 11:47:57.517369 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application result = self._query(query) Oct 11 11:47:57.517369 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 11 11:47:57.517369 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application conn.query(q) Oct 11 11:47:57.517369 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 11 11:47:57.517369 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 11 11:47:57.517369 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 11 11:47:57.517369 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application result.read() Oct 11 11:47:57.520231 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 11 11:47:57.520231 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 11 11:47:57.520231 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 11 11:47:57.520231 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 11 11:47:57.520231 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 11 11:47:57.520231 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 11 11:47:57.520231 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 11 11:47:57.520231 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 11 11:47:57.520231 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 11 11:47:57.520231 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:47:57.520231 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 11 11:47:57.520231 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:47:57.520231 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:47:57.520231 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 11 11:47:57.520231 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 11 11:47:57.520231 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 11 11:47:57.520231 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 11 11:47:57.520231 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 11 11:47:57.522465 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application next(self.gen) Oct 11 11:47:57.522465 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 11 11:47:57.522465 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application yield resource Oct 11 11:47:57.522465 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 11 11:47:57.522465 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application next(self.gen) Oct 11 11:47:57.522465 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 11 11:47:57.522465 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application self.session.rollback() Oct 11 11:47:57.522465 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 11 11:47:57.522465 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application self.force_reraise() Oct 11 11:47:57.522465 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 11 11:47:57.522465 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application raise self.value Oct 11 11:47:57.522465 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 11 11:47:57.522465 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 11 11:47:57.522465 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 11 11:47:57.522465 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application session.commit() Oct 11 11:47:57.522465 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 11 11:47:57.522465 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 11 11:47:57.524486 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 11 11:47:57.524486 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application self._prepare_impl() Oct 11 11:47:57.524486 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 11 11:47:57.524486 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application self.session.flush() Oct 11 11:47:57.524486 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 11 11:47:57.524486 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application self._flush(objects) Oct 11 11:47:57.524486 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 11 11:47:57.524486 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 11 11:47:57.524486 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 11 11:47:57.524486 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application compat.raise_( Oct 11 11:47:57.524486 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 11 11:47:57.524486 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application raise exception Oct 11 11:47:57.524486 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 11 11:47:57.524486 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application flush_context.execute() Oct 11 11:47:57.524486 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 11 11:47:57.524486 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application rec.execute(self) Oct 11 11:47:57.524486 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 11 11:47:57.527533 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 11 11:47:57.527533 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 11 11:47:57.527533 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 11 11:47:57.527533 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 11 11:47:57.527533 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 11 11:47:57.527533 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 11 11:47:57.527533 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 11 11:47:57.527533 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 11 11:47:57.527533 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 11 11:47:57.527533 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 11 11:47:57.527533 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 11 11:47:57.527533 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 11 11:47:57.527533 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 11 11:47:57.527533 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 11 11:47:57.527533 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 11 11:47:57.529606 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 11 11:47:57.529606 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application raise exception Oct 11 11:47:57.529606 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 11 11:47:57.529606 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 11 11:47:57.529606 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 11 11:47:57.529606 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 11 11:47:57.529606 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 11 11:47:57.529606 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application result = self._query(query) Oct 11 11:47:57.529606 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 11 11:47:57.529606 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application conn.query(q) Oct 11 11:47:57.529606 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 11 11:47:57.529606 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 11 11:47:57.529606 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 11 11:47:57.529606 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application result.read() Oct 11 11:47:57.529606 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 11 11:47:57.529606 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 11 11:47:57.529606 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 11 11:47:57.531665 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 11 11:47:57.531665 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 11 11:47:57.531665 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 11 11:47:57.531665 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 11 11:47:57.531665 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 11 11:47:57.531665 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 11 11:47:57.531665 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 11 11:47:57.531665 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application [parameters: {'id': 'cf52aaa93edf45f3b25fb5165173951a', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 11 11:47:57.531665 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 11 11:47:57.531665 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:47:57.531665 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 11 11:47:57.531665 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:47:57.531665 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:47:57.531665 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 11 11:47:57.531665 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 11 11:47:57.533402 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 11 11:47:57.533402 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 11 11:47:57.533402 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name Default. Oct 11 11:47:57.533402 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:47:57.533402 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 11 11:47:57.533402 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:47:57.533402 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:47:57.533402 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:47:57.533402 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:47:57.533402 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:47:57.533402 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:47:57.533402 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:47:57.533402 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:47:57.533402 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:47:57.533402 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:47:57.533402 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:47:57.533402 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:47:57.535864 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 128, in post Oct 11 11:47:57.535864 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_domain( Oct 11 11:47:57.535864 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:47:57.535864 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:47:57.535864 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 765, in create_domain Oct 11 11:47:57.535864 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application is_domain_project = self.create_project( Oct 11 11:47:57.535864 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:47:57.535864 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:47:57.535864 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 11 11:47:57.535864 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 11 11:47:57.535864 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 11 11:47:57.535864 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:47:57.535864 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 396/786] 10.222.0.59 () {60 vars in 1137 bytes} [Tue Oct 11 11:47:57 2022] POST /identity/v3/domains => generated 190 bytes in 230 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Oct 11 11:47:57.535864 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc68860-0c3e-4312-9ed6-b032c0f9a72a None None] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:57.537410 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc68860-0c3e-4312-9ed6-b032c0f9a72a None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:57.537410 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc68860-0c3e-4312-9ed6-b032c0f9a72a None None] PATH_INFO: `/` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:57.537410 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 397/787] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:47:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:47:57.537410 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc68860-0c3e-4312-9ed6-b032c0f9a72a None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:57.541411 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5851f72e-1fb8-44c5-a027-24f7d7b87baa None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:57.566263 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:57.591293 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e009de3-117a-45ce-8cba-44f9f246cde5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:57.592242 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e009de3-117a-45ce-8cba-44f9f246cde5 None admin] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:57.592700 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e009de3-117a-45ce-8cba-44f9f246cde5 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:57.593097 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e009de3-117a-45ce-8cba-44f9f246cde5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:57.594306 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e009de3-117a-45ce-8cba-44f9f246cde5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:57.596508 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e009de3-117a-45ce-8cba-44f9f246cde5 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:57.626406 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 391/788] 10.222.0.59 () {60 vars in 1144 bytes} [Tue Oct 11 11:47:57 2022] POST /identity/v3/endpoints => generated 334 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 11:47:57.632386 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13334e75-3e97-4a66-9d8f-5c3205c012dd None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:57.632821 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13334e75-3e97-4a66-9d8f-5c3205c012dd None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:57.633184 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13334e75-3e97-4a66-9d8f-5c3205c012dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:57.657228 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-13334e75-3e97-4a66-9d8f-5c3205c012dd None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:47:57.659237 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-13334e75-3e97-4a66-9d8f-5c3205c012dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:47:57.709773 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc68860-0c3e-4312-9ed6-b032c0f9a72a None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:57.744915 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 392/789] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:47:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 11:47:57.777576 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1686bf96-16be-49ae-9f3a-a5c247b77d88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:57.779034 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1686bf96-16be-49ae-9f3a-a5c247b77d88 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:57.779690 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1686bf96-16be-49ae-9f3a-a5c247b77d88 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:57.780122 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1686bf96-16be-49ae-9f3a-a5c247b77d88 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:57.794771 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1686bf96-16be-49ae-9f3a-a5c247b77d88 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:57.796315 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1686bf96-16be-49ae-9f3a-a5c247b77d88 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:57.815390 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application [None req-1686bf96-16be-49ae-9f3a-a5c247b77d88 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 11:47:57.815390 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:47:57.815390 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:47:57.815390 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:47:57.815390 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:47:57.815390 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:47:57.815390 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:47:57.815390 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:47:57.815390 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:47:57.815390 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:47:57.815390 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:47:57.815390 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:47:57.815390 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 11:47:57.815390 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 11:47:57.815390 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 11:47:57.815390 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 11:47:57.817156 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:47:57.817156 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:47:57.817156 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:47:57.817156 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:47:57.817156 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:47:57.817156 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:47:57.817156 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:47:57.817156 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application with Lock( Oct 11 11:47:57.817156 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:47:57.817156 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:47:57.817156 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:47:57.817156 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:47:57.817156 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:47:57.817156 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:47:57.817156 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:47:57.817156 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:47:57.817156 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 11:47:57.818934 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 11:47:57.818934 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 11:47:57.818934 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:47:57.818934 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 398/790] 10.222.0.59 () {56 vars in 1108 bytes} [Tue Oct 11 11:47:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:47:57.822728 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13334e75-3e97-4a66-9d8f-5c3205c012dd None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:57.836380 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:57.983610 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13334e75-3e97-4a66-9d8f-5c3205c012dd None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:58.014016 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1686bf96-16be-49ae-9f3a-a5c247b77d88 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:58.048611 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bea4e1-16fd-4c55-aa89-c2292bf19700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:58.049701 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bea4e1-16fd-4c55-aa89-c2292bf19700 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:58.050190 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bea4e1-16fd-4c55-aa89-c2292bf19700 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:58.050690 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bea4e1-16fd-4c55-aa89-c2292bf19700 None admin] PATH_INFO: `/v3/domains` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:58.051872 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bea4e1-16fd-4c55-aa89-c2292bf19700 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:47:58.052419 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bea4e1-16fd-4c55-aa89-c2292bf19700 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:58.055319 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bea4e1-16fd-4c55-aa89-c2292bf19700 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:58.082786 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 393/791] 10.222.0.59 () {56 vars in 1109 bytes} [Tue Oct 11 11:47:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:47:58.111309 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:58.130513 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1686bf96-16be-49ae-9f3a-a5c247b77d88 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:58.177311 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9fb67f-913b-48b8-a542-bf270e0917ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:58.177989 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9fb67f-913b-48b8-a542-bf270e0917ce None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:58.178268 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9fb67f-913b-48b8-a542-bf270e0917ce None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:58.178542 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9fb67f-913b-48b8-a542-bf270e0917ce None admin] PATH_INFO: `/v3/domains/default` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:58.179805 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9fb67f-913b-48b8-a542-bf270e0917ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:58.181453 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9fb67f-913b-48b8-a542-bf270e0917ce None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:58.183378 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 399/792] 10.222.0.59 () {56 vars in 1108 bytes} [Tue Oct 11 11:47:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 11:47:58.189310 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bea4e1-16fd-4c55-aa89-c2292bf19700 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:47:58.273592 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:47:58.313561 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bea4e1-16fd-4c55-aa89-c2292bf19700 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:47:58.353289 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15694a06-0484-4b33-9ddd-548faf4f079a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:47:58.354112 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15694a06-0484-4b33-9ddd-548faf4f079a None admin] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:58.354198 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15694a06-0484-4b33-9ddd-548faf4f079a None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:58.354434 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15694a06-0484-4b33-9ddd-548faf4f079a None admin] PATH_INFO: `/v3/projects` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:58.355147 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15694a06-0484-4b33-9ddd-548faf4f079a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:47:58.356603 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15694a06-0484-4b33-9ddd-548faf4f079a None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:47:58.400176 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 394/793] 10.222.0.59 () {60 vars in 1141 bytes} [Tue Oct 11 11:47:58 2022] POST /identity/v3/projects => generated 311 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 11:47:59.927784 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a4f006-6407-44a1-a2ed-9aefc92fc141 None None] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:59.928356 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a4f006-6407-44a1-a2ed-9aefc92fc141 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:59.928356 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a4f006-6407-44a1-a2ed-9aefc92fc141 None None] PATH_INFO: `/` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:59.929656 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 400/794] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:47:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:47:59.936384 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce93dba3-857a-4bb0-84e7-efdb12babf2d None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:47:59.936985 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce93dba3-857a-4bb0-84e7-efdb12babf2d None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:47:59.937483 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce93dba3-857a-4bb0-84e7-efdb12babf2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:47:59.968613 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-ce93dba3-857a-4bb0-84e7-efdb12babf2d None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:47:59.969991 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-ce93dba3-857a-4bb0-84e7-efdb12babf2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:48:00.043775 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 395/795] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:47:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 11:48:00.397807 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a4f006-6407-44a1-a2ed-9aefc92fc141 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:48:00.525914 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a4f006-6407-44a1-a2ed-9aefc92fc141 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:48:00.574547 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f2678a-1898-4df9-95e4-2aaf2db95b92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:48:00.575229 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f2678a-1898-4df9-95e4-2aaf2db95b92 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:48:00.575512 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f2678a-1898-4df9-95e4-2aaf2db95b92 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:48:00.575783 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f2678a-1898-4df9-95e4-2aaf2db95b92 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:48:00.585273 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:48:00.589304 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f2678a-1898-4df9-95e4-2aaf2db95b92 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 11:48:00.594966 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application [None req-b4f2678a-1898-4df9-95e4-2aaf2db95b92 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 11 11:48:00.594966 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:48:00.594966 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:48:00.594966 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:48:00.594966 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:48:00.594966 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:48:00.594966 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:48:00.594966 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:48:00.594966 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:48:00.594966 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:48:00.594966 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:48:00.594966 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:48:00.594966 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 11:48:00.594966 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 11:48:00.594966 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 11:48:00.594966 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 11:48:00.597146 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 11:48:00.597146 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 11:48:00.597146 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 11:48:00.597146 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 11:48:00.597146 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:48:00.597146 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:48:00.597146 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:48:00.597146 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:48:00.597146 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:48:00.597146 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:48:00.597146 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:48:00.597146 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application with Lock( Oct 11 11:48:00.597146 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:48:00.597146 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:48:00.597146 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:48:00.597146 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:48:00.598549 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:48:00.598549 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:48:00.598549 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:48:00.598549 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:48:00.598549 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 11:48:00.598549 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 11:48:00.598549 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 11:48:00.598549 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 11:48:00.598549 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 11:48:00.598549 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 11:48:00.598549 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 11 11:48:00.598549 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:48:00.598549 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 401/796] 10.222.0.59 () {56 vars in 1120 bytes} [Tue Oct 11 11:48:00 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:48:00.602780 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce93dba3-857a-4bb0-84e7-efdb12babf2d None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:48:00.646619 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55ccaf7-5024-45c8-a5d1-d262800e11cb None None] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:48:00.646903 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55ccaf7-5024-45c8-a5d1-d262800e11cb None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:48:00.647194 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55ccaf7-5024-45c8-a5d1-d262800e11cb None None] PATH_INFO: `/` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:48:00.648645 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 402/797] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:48:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 11 11:48:00.654465 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5db3be-e0f1-47de-9cd8-28eca315f5b8 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:48:00.654773 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5db3be-e0f1-47de-9cd8-28eca315f5b8 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:48:00.655051 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5db3be-e0f1-47de-9cd8-28eca315f5b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:48:00.664747 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:48:00.675439 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-1e5db3be-e0f1-47de-9cd8-28eca315f5b8 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:48:00.676646 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-1e5db3be-e0f1-47de-9cd8-28eca315f5b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:48:00.718255 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce93dba3-857a-4bb0-84e7-efdb12babf2d None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:48:00.747718 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 403/798] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:48:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 11 11:48:00.755144 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ded3820-78bd-4b4a-afe1-c1aa00b721cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:48:00.755764 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ded3820-78bd-4b4a-afe1-c1aa00b721cb None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:48:00.756040 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ded3820-78bd-4b4a-afe1-c1aa00b721cb None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:48:00.756308 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ded3820-78bd-4b4a-afe1-c1aa00b721cb None admin] PATH_INFO: `/v3/roles` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:48:00.756975 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ded3820-78bd-4b4a-afe1-c1aa00b721cb None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:48:00.757273 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ded3820-78bd-4b4a-afe1-c1aa00b721cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:48:00.758667 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ded3820-78bd-4b4a-afe1-c1aa00b721cb None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:48:00.767676 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 396/799] 10.222.0.59 () {56 vars in 1115 bytes} [Tue Oct 11 11:48:00 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:48:00.775462 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5db3be-e0f1-47de-9cd8-28eca315f5b8 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:48:00.888902 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5db3be-e0f1-47de-9cd8-28eca315f5b8 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:48:00.922054 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:48:00.931615 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28030ddf-68a8-4345-aa4d-d7dfa7eebe6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:48:00.932344 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28030ddf-68a8-4345-aa4d-d7dfa7eebe6e None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:48:00.932549 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28030ddf-68a8-4345-aa4d-d7dfa7eebe6e None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:48:00.932824 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28030ddf-68a8-4345-aa4d-d7dfa7eebe6e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:48:00.944047 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28030ddf-68a8-4345-aa4d-d7dfa7eebe6e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:48:00.945517 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28030ddf-68a8-4345-aa4d-d7dfa7eebe6e None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:48:00.960944 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application [None req-28030ddf-68a8-4345-aa4d-d7dfa7eebe6e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 11:48:00.960944 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:48:00.960944 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:48:00.960944 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:48:00.960944 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:48:00.960944 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:48:00.960944 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:48:00.960944 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:48:00.960944 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:48:00.960944 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:48:00.960944 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:48:00.960944 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:48:00.960944 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 11:48:00.960944 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 11:48:00.960944 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 11:48:00.960944 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 11:48:00.961961 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:48:00.961961 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:48:00.961961 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:48:00.961961 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:48:00.961961 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:48:00.961961 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:48:00.961961 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:48:00.961961 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application with Lock( Oct 11 11:48:00.961961 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:48:00.961961 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:48:00.961961 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:48:00.961961 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:48:00.961961 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:48:00.961961 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:48:00.961961 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:48:00.961961 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:48:00.961961 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 11:48:00.967752 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 11:48:00.967752 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 11:48:00.967752 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:48:00.967752 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 404/800] 10.222.0.59 () {56 vars in 1108 bytes} [Tue Oct 11 11:48:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:48:00.968516 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ded3820-78bd-4b4a-afe1-c1aa00b721cb None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:48:01.026423 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28030ddf-68a8-4345-aa4d-d7dfa7eebe6e None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:48:01.098378 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ded3820-78bd-4b4a-afe1-c1aa00b721cb None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:48:01.114444 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:48:01.140556 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62e9edc-b0a1-4bb8-a7cc-b49ac1d50fc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:48:01.141278 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62e9edc-b0a1-4bb8-a7cc-b49ac1d50fc1 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:48:01.141563 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62e9edc-b0a1-4bb8-a7cc-b49ac1d50fc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:48:01.141835 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62e9edc-b0a1-4bb8-a7cc-b49ac1d50fc1 None admin] PATH_INFO: `/v3/domains` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:48:01.142504 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62e9edc-b0a1-4bb8-a7cc-b49ac1d50fc1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:48:01.142842 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62e9edc-b0a1-4bb8-a7cc-b49ac1d50fc1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:48:01.144267 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62e9edc-b0a1-4bb8-a7cc-b49ac1d50fc1 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:48:01.154425 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28030ddf-68a8-4345-aa4d-d7dfa7eebe6e None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:48:01.158407 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 397/801] 10.222.0.59 () {56 vars in 1109 bytes} [Tue Oct 11 11:48:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:48:01.167735 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62e9edc-b0a1-4bb8-a7cc-b49ac1d50fc1 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:48:01.200291 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349e6c0d-5d7e-430e-9e91-8205e6fac578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:48:01.200869 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349e6c0d-5d7e-430e-9e91-8205e6fac578 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:48:01.201151 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349e6c0d-5d7e-430e-9e91-8205e6fac578 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:48:01.201441 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349e6c0d-5d7e-430e-9e91-8205e6fac578 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:48:01.202659 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349e6c0d-5d7e-430e-9e91-8205e6fac578 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:48:01.204293 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349e6c0d-5d7e-430e-9e91-8205e6fac578 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:48:01.206216 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 405/802] 10.222.0.59 () {56 vars in 1108 bytes} [Tue Oct 11 11:48:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:48:01.214968 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349e6c0d-5d7e-430e-9e91-8205e6fac578 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:48:01.276633 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:48:01.300760 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62e9edc-b0a1-4bb8-a7cc-b49ac1d50fc1 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:48:01.334104 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349e6c0d-5d7e-430e-9e91-8205e6fac578 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:48:01.345967 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13803c9a-95a3-4539-befa-35be96add3cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:48:01.346721 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13803c9a-95a3-4539-befa-35be96add3cc None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:48:01.347095 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13803c9a-95a3-4539-befa-35be96add3cc None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:48:01.347375 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13803c9a-95a3-4539-befa-35be96add3cc None admin] PATH_INFO: `/v3/users/nova` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:48:01.355376 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13803c9a-95a3-4539-befa-35be96add3cc None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:48:01.356850 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13803c9a-95a3-4539-befa-35be96add3cc None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:48:01.369361 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application [None req-13803c9a-95a3-4539-befa-35be96add3cc None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 11 11:48:01.369361 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:48:01.369361 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:48:01.369361 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:48:01.369361 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:48:01.369361 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:48:01.369361 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:48:01.369361 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:48:01.369361 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:48:01.369361 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:48:01.369361 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:48:01.369361 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:48:01.369361 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 11 11:48:01.369361 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 11 11:48:01.369361 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 11 11:48:01.369361 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 11 11:48:01.370547 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:48:01.370547 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:48:01.370547 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 11 11:48:01.370547 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 11:48:01.370547 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 11 11:48:01.370547 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 11:48:01.370547 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:48:01.370547 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:48:01.370547 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:48:01.370547 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:48:01.370547 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:48:01.370547 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application with Lock( Oct 11 11:48:01.370547 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:48:01.370547 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:48:01.370547 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:48:01.370547 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:48:01.370547 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:48:01.370547 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:48:01.371894 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:48:01.371894 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:48:01.371894 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 11 11:48:01.371894 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 11 11:48:01.371894 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 11 11:48:01.371894 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 11 11:48:01.371894 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 11 11:48:01.371894 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 11 11:48:01.371894 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 11 11:48:01.371894 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:48:01.371894 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 398/803] 10.222.0.59 () {56 vars in 1093 bytes} [Tue Oct 11 11:48:01 2022] GET /identity/v3/users/nova => generated 82 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:48:01.376229 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc86a285-7c83-472b-ac1a-14dc8a95fbac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:48:01.376522 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13803c9a-95a3-4539-befa-35be96add3cc None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:48:01.377226 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc86a285-7c83-472b-ac1a-14dc8a95fbac None admin] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:48:01.377748 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc86a285-7c83-472b-ac1a-14dc8a95fbac None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:48:01.378228 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc86a285-7c83-472b-ac1a-14dc8a95fbac None admin] PATH_INFO: `/v3/users` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:48:01.379357 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc86a285-7c83-472b-ac1a-14dc8a95fbac None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:48:01.382148 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc86a285-7c83-472b-ac1a-14dc8a95fbac None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:48:01.411378 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 406/804] 10.222.0.59 () {60 vars in 1132 bytes} [Tue Oct 11 11:48:01 2022] POST /identity/v3/users => generated 285 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 11:48:01.415713 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:48:01.506532 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13803c9a-95a3-4539-befa-35be96add3cc None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:48:01.549578 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f83816-0697-4501-be05-f69178800091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:48:01.550339 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f83816-0697-4501-be05-f69178800091 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:48:01.550723 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f83816-0697-4501-be05-f69178800091 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:48:01.551047 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f83816-0697-4501-be05-f69178800091 None admin] PATH_INFO: `/v3/users` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:48:01.551898 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f83816-0697-4501-be05-f69178800091 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:48:01.552376 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f83816-0697-4501-be05-f69178800091 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:48:01.554311 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f83816-0697-4501-be05-f69178800091 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:48:01.578529 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 399/805] 10.222.0.59 () {56 vars in 1133 bytes} [Tue Oct 11 11:48:01 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:48:01.585338 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc86a285-7c83-472b-ac1a-14dc8a95fbac None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:48:01.720367 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc86a285-7c83-472b-ac1a-14dc8a95fbac None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:48:01.728769 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:48:01.761316 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6643223-4435-4d61-bc60-8dc469374b19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:48:01.761888 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6643223-4435-4d61-bc60-8dc469374b19 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:48:01.762169 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6643223-4435-4d61-bc60-8dc469374b19 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:48:01.762443 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6643223-4435-4d61-bc60-8dc469374b19 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:48:01.774377 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6643223-4435-4d61-bc60-8dc469374b19 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:48:01.775909 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6643223-4435-4d61-bc60-8dc469374b19 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:48:01.792415 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application [None req-a6643223-4435-4d61-bc60-8dc469374b19 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 11:48:01.792415 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:48:01.792415 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:48:01.792415 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:48:01.792415 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:48:01.792415 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:48:01.792415 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:48:01.792415 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:48:01.792415 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:48:01.792415 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:48:01.792415 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:48:01.792415 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:48:01.792415 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 11:48:01.792415 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 11:48:01.792415 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 11:48:01.792415 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 11:48:01.793519 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:48:01.793519 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:48:01.793519 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:48:01.793519 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:48:01.793519 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:48:01.793519 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:48:01.793519 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:48:01.793519 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application with Lock( Oct 11 11:48:01.793519 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:48:01.793519 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:48:01.793519 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:48:01.793519 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:48:01.793519 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:48:01.793519 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:48:01.793519 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:48:01.793519 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:48:01.793519 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 11:48:01.794412 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 11:48:01.794412 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 11:48:01.794412 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:48:01.794412 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 407/806] 10.222.0.59 () {56 vars in 1108 bytes} [Tue Oct 11 11:48:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:48:01.799764 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f83816-0697-4501-be05-f69178800091 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:48:01.897395 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:48:01.908000 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f83816-0697-4501-be05-f69178800091 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:48:01.946230 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3108f169-e487-4648-92f8-f04d457ba9f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:48:01.946884 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3108f169-e487-4648-92f8-f04d457ba9f1 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:48:01.947167 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3108f169-e487-4648-92f8-f04d457ba9f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:48:01.947437 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3108f169-e487-4648-92f8-f04d457ba9f1 None admin] PATH_INFO: `/v3/domains` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:48:01.948098 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3108f169-e487-4648-92f8-f04d457ba9f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:48:01.948403 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3108f169-e487-4648-92f8-f04d457ba9f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:48:01.949772 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3108f169-e487-4648-92f8-f04d457ba9f1 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:48:01.966586 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 400/807] 10.222.0.59 () {56 vars in 1109 bytes} [Tue Oct 11 11:48:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 11:48:01.968023 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6643223-4435-4d61-bc60-8dc469374b19 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:48:02.075571 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6643223-4435-4d61-bc60-8dc469374b19 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:48:02.110211 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a255cb28-b9a6-4e98-bd1c-249e36d55639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:48:02.110889 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a255cb28-b9a6-4e98-bd1c-249e36d55639 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:48:02.111180 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a255cb28-b9a6-4e98-bd1c-249e36d55639 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:48:02.111444 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a255cb28-b9a6-4e98-bd1c-249e36d55639 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:48:02.119289 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a255cb28-b9a6-4e98-bd1c-249e36d55639 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:48:02.120760 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a255cb28-b9a6-4e98-bd1c-249e36d55639 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:48:02.132769 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application [None req-a255cb28-b9a6-4e98-bd1c-249e36d55639 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 11:48:02.132769 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:48:02.132769 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:48:02.132769 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:48:02.132769 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:48:02.132769 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:48:02.132769 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:48:02.132769 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:48:02.132769 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:48:02.132769 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:48:02.132769 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:48:02.132769 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:48:02.132769 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 11 11:48:02.132769 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 11 11:48:02.132769 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 11 11:48:02.133718 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 11 11:48:02.133718 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:48:02.133718 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:48:02.133718 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:48:02.133718 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:48:02.133718 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:48:02.133718 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:48:02.133718 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:48:02.133718 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application with Lock( Oct 11 11:48:02.133718 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:48:02.133718 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:48:02.133718 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:48:02.133718 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:48:02.133718 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:48:02.133718 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:48:02.133718 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:48:02.133718 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:48:02.134637 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 11 11:48:02.134637 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 11 11:48:02.134637 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 11 11:48:02.134637 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 11 11:48:02.134637 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 11 11:48:02.134637 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 11 11:48:02.134637 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 11:48:02.134637 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:48:02.134637 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 408/808] 10.222.0.59 () {56 vars in 1111 bytes} [Tue Oct 11 11:48:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:48:02.139945 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3108f169-e487-4648-92f8-f04d457ba9f1 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:48:02.247466 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3108f169-e487-4648-92f8-f04d457ba9f1 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:48:02.282644 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f90fb5-4ac4-404f-a854-f8f045092dfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:48:02.283289 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f90fb5-4ac4-404f-a854-f8f045092dfd None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:48:02.283565 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f90fb5-4ac4-404f-a854-f8f045092dfd None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:48:02.284127 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f90fb5-4ac4-404f-a854-f8f045092dfd None admin] PATH_INFO: `/v3/projects` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:48:02.284813 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f90fb5-4ac4-404f-a854-f8f045092dfd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:48:02.285113 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f90fb5-4ac4-404f-a854-f8f045092dfd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:48:02.286526 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f90fb5-4ac4-404f-a854-f8f045092dfd None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:48:02.299543 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 401/809] 10.222.0.59 () {56 vars in 1148 bytes} [Tue Oct 11 11:48:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:48:02.305674 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a255cb28-b9a6-4e98-bd1c-249e36d55639 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:48:02.326218 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:48:02.407743 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a255cb28-b9a6-4e98-bd1c-249e36d55639 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:48:02.443539 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3ecfa6-ef98-4d77-938e-c5f4ac01ee5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:48:02.444190 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3ecfa6-ef98-4d77-938e-c5f4ac01ee5e None admin] REQUEST_METHOD: `PUT` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:48:02.444467 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3ecfa6-ef98-4d77-938e-c5f4ac01ee5e None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:48:02.444739 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3ecfa6-ef98-4d77-938e-c5f4ac01ee5e None admin] PATH_INFO: `/v3/projects/26ecf552cc9a4e5ca1d380509683dc4a/users/bad5e1cb306b46818096940acc3810b2/roles/6a95127666ae4707bc09aca24e0d3d39` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:48:02.446949 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3ecfa6-ef98-4d77-938e-c5f4ac01ee5e None admin] RBAC: Authorizing `identity:create_grant(project_id=26ecf552cc9a4e5ca1d380509683dc4a, user_id=bad5e1cb306b46818096940acc3810b2, role_id=6a95127666ae4707bc09aca24e0d3d39)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:48:02.448389 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3ecfa6-ef98-4d77-938e-c5f4ac01ee5e None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:48:02.462894 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 409/810] 10.222.0.59 () {58 vars in 1439 bytes} [Tue Oct 11 11:48:02 2022] PUT /identity/v3/projects/26ecf552cc9a4e5ca1d380509683dc4a/users/bad5e1cb306b46818096940acc3810b2/roles/6a95127666ae4707bc09aca24e0d3d39 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 11:48:04.631950 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d88113f-9eec-4164-90f5-24952a177a1e None None] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:48:04.632533 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d88113f-9eec-4164-90f5-24952a177a1e None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:48:04.632533 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d88113f-9eec-4164-90f5-24952a177a1e None None] PATH_INFO: `/` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:48:04.633897 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 402/811] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:48:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:48:04.640047 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f56940-950b-4292-bf13-6d2590e43cf8 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:48:04.640729 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f56940-950b-4292-bf13-6d2590e43cf8 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:48:04.640729 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f56940-950b-4292-bf13-6d2590e43cf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:48:04.661142 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-21f56940-950b-4292-bf13-6d2590e43cf8 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:48:04.662359 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-21f56940-950b-4292-bf13-6d2590e43cf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:48:04.736874 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 410/812] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:48:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 11:48:05.002971 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d88113f-9eec-4164-90f5-24952a177a1e None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:48:05.019560 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:48:05.082177 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da35161b-3595-4b85-83ba-641025083023 None None] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:48:05.082724 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da35161b-3595-4b85-83ba-641025083023 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:48:05.083233 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da35161b-3595-4b85-83ba-641025083023 None None] PATH_INFO: `/` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:48:05.085669 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 411/813] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:48:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:48:05.094340 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a15f57f-87b3-47f5-816a-ce984d5bc5f0 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:48:05.094874 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a15f57f-87b3-47f5-816a-ce984d5bc5f0 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:48:05.095360 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a15f57f-87b3-47f5-816a-ce984d5bc5f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:48:05.114687 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d88113f-9eec-4164-90f5-24952a177a1e None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:48:05.124023 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-9a15f57f-87b3-47f5-816a-ce984d5bc5f0 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:48:05.125426 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-9a15f57f-87b3-47f5-816a-ce984d5bc5f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:48:05.127736 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:48:05.153217 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599c690a-0591-492b-b92f-8fe39d59c356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:48:05.153862 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599c690a-0591-492b-b92f-8fe39d59c356 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:48:05.154148 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599c690a-0591-492b-b92f-8fe39d59c356 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:48:05.154411 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599c690a-0591-492b-b92f-8fe39d59c356 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:48:05.165709 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599c690a-0591-492b-b92f-8fe39d59c356 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 11:48:05.170503 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application [None req-599c690a-0591-492b-b92f-8fe39d59c356 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 11 11:48:05.170503 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:48:05.170503 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:48:05.170503 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:48:05.170503 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:48:05.170503 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:48:05.170503 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:48:05.170503 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:48:05.170503 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:48:05.170503 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:48:05.170503 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:48:05.170503 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:48:05.170503 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 11:48:05.170503 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 11:48:05.170503 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 11:48:05.170503 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 11:48:05.171746 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 11:48:05.171746 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 11:48:05.171746 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 11:48:05.171746 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 11:48:05.171746 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:48:05.171746 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:48:05.171746 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:48:05.171746 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:48:05.171746 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:48:05.171746 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:48:05.171746 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:48:05.171746 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application with Lock( Oct 11 11:48:05.171746 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:48:05.171746 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:48:05.171746 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:48:05.171746 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:48:05.172593 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:48:05.172593 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:48:05.172593 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:48:05.172593 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:48:05.172593 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 11:48:05.172593 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 11:48:05.172593 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 11:48:05.172593 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 11:48:05.172593 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 11:48:05.172593 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 11:48:05.172593 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 11 11:48:05.172593 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:48:05.172593 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 403/814] 10.222.0.59 () {56 vars in 1120 bytes} [Tue Oct 11 11:48:05 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:48:05.177935 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599c690a-0591-492b-b92f-8fe39d59c356 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:48:05.198333 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 412/815] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:48:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 11:48:05.251247 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:48:05.282415 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599c690a-0591-492b-b92f-8fe39d59c356 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:48:05.317887 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a685160-3792-4b8a-93a0-96b6728effaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:48:05.318487 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a685160-3792-4b8a-93a0-96b6728effaf None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:48:05.318860 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a685160-3792-4b8a-93a0-96b6728effaf None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:48:05.319055 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a685160-3792-4b8a-93a0-96b6728effaf None admin] PATH_INFO: `/v3/roles` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:48:05.319712 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a685160-3792-4b8a-93a0-96b6728effaf None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:48:05.320095 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a685160-3792-4b8a-93a0-96b6728effaf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:48:05.321461 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a685160-3792-4b8a-93a0-96b6728effaf None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:48:05.329552 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 404/816] 10.222.0.59 () {56 vars in 1115 bytes} [Tue Oct 11 11:48:05 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 11:48:05.335528 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a15f57f-87b3-47f5-816a-ce984d5bc5f0 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:48:05.438592 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a15f57f-87b3-47f5-816a-ce984d5bc5f0 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:48:05.460181 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a685160-3792-4b8a-93a0-96b6728effaf None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:48:05.475342 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6750499e-c7c4-47b4-9124-044d7efacfc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:48:05.475982 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6750499e-c7c4-47b4-9124-044d7efacfc6 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:48:05.476260 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6750499e-c7c4-47b4-9124-044d7efacfc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:48:05.476538 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6750499e-c7c4-47b4-9124-044d7efacfc6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:48:05.483372 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:48:05.493159 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6750499e-c7c4-47b4-9124-044d7efacfc6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:48:05.495353 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6750499e-c7c4-47b4-9124-044d7efacfc6 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:48:05.511459 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application [None req-6750499e-c7c4-47b4-9124-044d7efacfc6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 11:48:05.511459 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:48:05.511459 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:48:05.511459 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:48:05.511459 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:48:05.511459 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:48:05.511459 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:48:05.511459 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:48:05.511459 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:48:05.511459 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:48:05.511459 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:48:05.511459 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:48:05.511459 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 11:48:05.511459 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 11:48:05.511459 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 11:48:05.511459 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 11:48:05.512471 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:48:05.512471 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:48:05.512471 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:48:05.512471 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:48:05.512471 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:48:05.512471 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:48:05.512471 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:48:05.512471 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application with Lock( Oct 11 11:48:05.512471 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:48:05.512471 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:48:05.512471 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:48:05.512471 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:48:05.512471 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:48:05.512471 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:48:05.512471 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:48:05.512471 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:48:05.512471 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 11:48:05.513357 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 11:48:05.513357 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 11:48:05.513357 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:48:05.513357 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 413/817] 10.222.0.59 () {56 vars in 1108 bytes} [Tue Oct 11 11:48:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:48:05.519011 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6750499e-c7c4-47b4-9124-044d7efacfc6 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:48:05.566587 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a685160-3792-4b8a-93a0-96b6728effaf None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:48:05.600050 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9c25d4-2e4b-475c-9c6c-243648841a6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:48:05.600677 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9c25d4-2e4b-475c-9c6c-243648841a6a None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:48:05.600953 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9c25d4-2e4b-475c-9c6c-243648841a6a None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:48:05.601225 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9c25d4-2e4b-475c-9c6c-243648841a6a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:48:05.603296 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:48:05.613804 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9c25d4-2e4b-475c-9c6c-243648841a6a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 11:48:05.618597 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application [None req-7d9c25d4-2e4b-475c-9c6c-243648841a6a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 11 11:48:05.618597 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:48:05.618597 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:48:05.618597 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:48:05.618597 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:48:05.618597 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:48:05.618597 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:48:05.618597 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:48:05.618597 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:48:05.618597 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:48:05.618597 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:48:05.618597 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:48:05.618597 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 11:48:05.618597 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 11:48:05.618597 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 11:48:05.618597 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 11:48:05.619563 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 11:48:05.619563 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 11:48:05.619563 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 11:48:05.619563 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 11:48:05.619563 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:48:05.619563 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:48:05.619563 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:48:05.619563 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:48:05.619563 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:48:05.619563 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:48:05.619563 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:48:05.619563 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application with Lock( Oct 11 11:48:05.619563 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:48:05.619563 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:48:05.619563 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:48:05.619563 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:48:05.620422 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:48:05.620422 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:48:05.620422 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:48:05.620422 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:48:05.620422 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 11:48:05.620422 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 11:48:05.620422 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 11:48:05.620422 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 11:48:05.620422 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 11:48:05.620422 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 11:48:05.620422 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 11 11:48:05.620422 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:48:05.620422 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 405/818] 10.222.0.59 () {56 vars in 1096 bytes} [Tue Oct 11 11:48:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:48:05.625783 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9c25d4-2e4b-475c-9c6c-243648841a6a None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:48:05.628107 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6750499e-c7c4-47b4-9124-044d7efacfc6 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:48:05.665676 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d1f074-1ce7-4509-af94-dbfa1ca5551c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:48:05.666734 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d1f074-1ce7-4509-af94-dbfa1ca5551c None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:48:05.667221 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d1f074-1ce7-4509-af94-dbfa1ca5551c None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:48:05.667714 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d1f074-1ce7-4509-af94-dbfa1ca5551c None admin] PATH_INFO: `/v3/domains` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:48:05.668880 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d1f074-1ce7-4509-af94-dbfa1ca5551c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:48:05.669436 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d1f074-1ce7-4509-af94-dbfa1ca5551c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:48:05.672073 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d1f074-1ce7-4509-af94-dbfa1ca5551c None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:48:05.693919 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 414/819] 10.222.0.59 () {56 vars in 1109 bytes} [Tue Oct 11 11:48:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 11:48:05.703264 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d1f074-1ce7-4509-af94-dbfa1ca5551c None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:48:05.734113 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9c25d4-2e4b-475c-9c6c-243648841a6a None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:48:05.756343 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:48:05.770406 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26b4006-f1f3-4dde-b22f-9722df265151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:48:05.770983 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26b4006-f1f3-4dde-b22f-9722df265151 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:48:05.771268 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26b4006-f1f3-4dde-b22f-9722df265151 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:48:05.771545 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26b4006-f1f3-4dde-b22f-9722df265151 None admin] PATH_INFO: `/v3/roles` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:48:05.772227 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26b4006-f1f3-4dde-b22f-9722df265151 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:48:05.772533 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26b4006-f1f3-4dde-b22f-9722df265151 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:48:05.773913 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26b4006-f1f3-4dde-b22f-9722df265151 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:48:05.783273 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 406/820] 10.222.0.59 () {56 vars in 1099 bytes} [Tue Oct 11 11:48:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:48:05.789908 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26b4006-f1f3-4dde-b22f-9722df265151 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:48:05.811126 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d1f074-1ce7-4509-af94-dbfa1ca5551c None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:48:05.846811 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67422ac-402c-40ab-b194-405d3e1b5e2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:48:05.847445 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67422ac-402c-40ab-b194-405d3e1b5e2f None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:48:05.847721 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67422ac-402c-40ab-b194-405d3e1b5e2f None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:48:05.848000 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67422ac-402c-40ab-b194-405d3e1b5e2f None admin] PATH_INFO: `/v3/users/nova` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:48:05.848141 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:48:05.856659 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67422ac-402c-40ab-b194-405d3e1b5e2f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:48:05.858156 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67422ac-402c-40ab-b194-405d3e1b5e2f None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:48:05.871115 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application [None req-e67422ac-402c-40ab-b194-405d3e1b5e2f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 11 11:48:05.871115 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:48:05.871115 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:48:05.871115 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:48:05.871115 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:48:05.871115 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:48:05.871115 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:48:05.871115 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:48:05.871115 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:48:05.871115 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:48:05.871115 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:48:05.871115 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:48:05.871115 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 11 11:48:05.871115 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 11 11:48:05.871115 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 11 11:48:05.871115 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 11 11:48:05.872823 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:48:05.872823 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:48:05.872823 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 11 11:48:05.872823 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 11:48:05.872823 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 11 11:48:05.872823 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 11:48:05.872823 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:48:05.872823 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:48:05.872823 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:48:05.872823 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:48:05.872823 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:48:05.872823 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application with Lock( Oct 11 11:48:05.872823 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:48:05.872823 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:48:05.872823 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:48:05.872823 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:48:05.872823 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:48:05.872823 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:48:05.874461 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:48:05.874461 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:48:05.874461 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 11 11:48:05.874461 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 11 11:48:05.874461 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 11 11:48:05.874461 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 11 11:48:05.874461 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 11 11:48:05.874461 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 11 11:48:05.874461 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 11 11:48:05.874461 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:48:05.874461 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 415/821] 10.222.0.59 () {56 vars in 1093 bytes} [Tue Oct 11 11:48:05 2022] GET /identity/v3/users/nova => generated 82 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:48:05.877874 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67422ac-402c-40ab-b194-405d3e1b5e2f None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:48:05.898591 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26b4006-f1f3-4dde-b22f-9722df265151 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:48:05.993831 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67422ac-402c-40ab-b194-405d3e1b5e2f None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:48:06.006072 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:48:06.031819 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3babe621-1f94-41f1-b151-62962d252803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:48:06.032354 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3babe621-1f94-41f1-b151-62962d252803 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:48:06.032635 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3babe621-1f94-41f1-b151-62962d252803 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:48:06.032915 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3babe621-1f94-41f1-b151-62962d252803 None admin] PATH_INFO: `/v3/users` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:48:06.033682 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3babe621-1f94-41f1-b151-62962d252803 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:48:06.034350 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3babe621-1f94-41f1-b151-62962d252803 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:48:06.035928 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3babe621-1f94-41f1-b151-62962d252803 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:48:06.057236 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 416/822] 10.222.0.59 () {56 vars in 1133 bytes} [Tue Oct 11 11:48:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 11:48:06.063976 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3babe621-1f94-41f1-b151-62962d252803 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:48:06.067902 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac0f2ab-22fb-432d-8130-771281bd36ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:48:06.068541 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac0f2ab-22fb-432d-8130-771281bd36ba None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:48:06.068817 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac0f2ab-22fb-432d-8130-771281bd36ba None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:48:06.069082 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac0f2ab-22fb-432d-8130-771281bd36ba None admin] PATH_INFO: `/v3/users/a133b2bca86240d0b751c42694520e12` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:48:06.083589 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac0f2ab-22fb-432d-8130-771281bd36ba None admin] RBAC: Authorizing `identity:get_user(user_id=a133b2bca86240d0b751c42694520e12)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:48:06.085096 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac0f2ab-22fb-432d-8130-771281bd36ba None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:48:06.086961 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 407/823] 10.222.0.59 () {56 vars in 1177 bytes} [Tue Oct 11 11:48:05 2022] GET /identity/v3/users/a133b2bca86240d0b751c42694520e12 => generated 285 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 11:48:06.093508 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac0f2ab-22fb-432d-8130-771281bd36ba None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:48:06.173158 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3babe621-1f94-41f1-b151-62962d252803 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:48:06.199490 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac0f2ab-22fb-432d-8130-771281bd36ba None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:48:06.210000 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f8670e-a94a-4b4e-8ac6-1dafb89b8cc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:48:06.210646 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f8670e-a94a-4b4e-8ac6-1dafb89b8cc5 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:48:06.210927 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f8670e-a94a-4b4e-8ac6-1dafb89b8cc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:48:06.211197 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f8670e-a94a-4b4e-8ac6-1dafb89b8cc5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:48:06.215018 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:48:06.237964 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd87361b-7cb0-404e-a828-2a98066633c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:48:06.238621 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd87361b-7cb0-404e-a828-2a98066633c3 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:48:06.238900 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd87361b-7cb0-404e-a828-2a98066633c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:48:06.239176 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd87361b-7cb0-404e-a828-2a98066633c3 None admin] PATH_INFO: `/v3/projects/f4196b8338fa43c1bd146eeef8bd2995` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:48:06.240421 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd87361b-7cb0-404e-a828-2a98066633c3 None admin] RBAC: Authorizing `identity:get_project(project_id=f4196b8338fa43c1bd146eeef8bd2995)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:48:06.241876 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd87361b-7cb0-404e-a828-2a98066633c3 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:48:06.243597 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 408/824] 10.222.0.59 () {56 vars in 1186 bytes} [Tue Oct 11 11:48:06 2022] GET /identity/v3/projects/f4196b8338fa43c1bd146eeef8bd2995 => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 11:48:06.247235 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:48:06.249820 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd87361b-7cb0-404e-a828-2a98066633c3 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:48:06.282586 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f8670e-a94a-4b4e-8ac6-1dafb89b8cc5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:48:06.284451 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f8670e-a94a-4b4e-8ac6-1dafb89b8cc5 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:48:06.356628 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd87361b-7cb0-404e-a828-2a98066633c3 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:48:06.392582 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d010caa-65a0-4d0c-915d-cd6f06037707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:48:06.393253 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d010caa-65a0-4d0c-915d-cd6f06037707 None admin] REQUEST_METHOD: `PUT` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:48:06.393534 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d010caa-65a0-4d0c-915d-cd6f06037707 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:48:06.393810 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d010caa-65a0-4d0c-915d-cd6f06037707 None admin] PATH_INFO: `/v3/projects/f4196b8338fa43c1bd146eeef8bd2995/users/a133b2bca86240d0b751c42694520e12/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:48:06.399172 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 417/825] 10.222.0.59 () {58 vars in 1282 bytes} [Tue Oct 11 11:48:06 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 11 11:48:06.399338 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d010caa-65a0-4d0c-915d-cd6f06037707 None admin] RBAC: Authorizing `identity:create_grant(project_id=f4196b8338fa43c1bd146eeef8bd2995, user_id=a133b2bca86240d0b751c42694520e12, role_id=5cf1f4bd2fef46ab9c6eff56d52f9f69)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:48:06.399338 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d010caa-65a0-4d0c-915d-cd6f06037707 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:48:06.400681 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f8670e-a94a-4b4e-8ac6-1dafb89b8cc5 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:48:06.412952 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 409/826] 10.222.0.59 () {58 vars in 1439 bytes} [Tue Oct 11 11:48:06 2022] PUT /identity/v3/projects/f4196b8338fa43c1bd146eeef8bd2995/users/a133b2bca86240d0b751c42694520e12/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 11:48:06.478039 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:48:06.520023 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f8670e-a94a-4b4e-8ac6-1dafb89b8cc5 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:48:06.563419 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ab75c4-3f31-4b94-b756-2515f517a886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:48:06.564078 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ab75c4-3f31-4b94-b756-2515f517a886 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:48:06.564357 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ab75c4-3f31-4b94-b756-2515f517a886 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:48:06.564633 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ab75c4-3f31-4b94-b756-2515f517a886 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:48:06.576350 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ab75c4-3f31-4b94-b756-2515f517a886 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:48:06.577846 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ab75c4-3f31-4b94-b756-2515f517a886 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:48:06.594828 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application [None req-b7ab75c4-3f31-4b94-b756-2515f517a886 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 11:48:06.594828 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:48:06.594828 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:48:06.594828 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:48:06.594828 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:48:06.594828 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:48:06.594828 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:48:06.594828 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:48:06.594828 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:48:06.594828 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:48:06.594828 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:48:06.594828 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:48:06.594828 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 11:48:06.594828 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 11:48:06.594828 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 11:48:06.594828 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 11:48:06.596004 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:48:06.596004 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:48:06.596004 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:48:06.596004 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:48:06.596004 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:48:06.596004 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:48:06.596004 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:48:06.596004 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application with Lock( Oct 11 11:48:06.596004 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:48:06.596004 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:48:06.596004 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:48:06.596004 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:48:06.596004 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:48:06.596004 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:48:06.596004 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:48:06.596004 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:48:06.596004 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 11:48:06.597009 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 11:48:06.597009 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 11:48:06.597009 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:48:06.597009 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 418/827] 10.222.0.59 () {56 vars in 1108 bytes} [Tue Oct 11 11:48:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:48:06.602889 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d010caa-65a0-4d0c-915d-cd6f06037707 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:48:06.655950 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:48:06.717988 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d010caa-65a0-4d0c-915d-cd6f06037707 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:48:06.753957 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9977953-98bb-4744-9ace-3a2e8de18aa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:48:06.754668 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9977953-98bb-4744-9ace-3a2e8de18aa1 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:48:06.754862 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9977953-98bb-4744-9ace-3a2e8de18aa1 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:48:06.755152 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9977953-98bb-4744-9ace-3a2e8de18aa1 None admin] PATH_INFO: `/v3/domains` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:48:06.755781 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9977953-98bb-4744-9ace-3a2e8de18aa1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:48:06.756077 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9977953-98bb-4744-9ace-3a2e8de18aa1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:48:06.757442 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9977953-98bb-4744-9ace-3a2e8de18aa1 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:48:06.770093 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 410/828] 10.222.0.59 () {56 vars in 1109 bytes} [Tue Oct 11 11:48:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:48:06.776097 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ab75c4-3f31-4b94-b756-2515f517a886 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:48:06.882417 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ab75c4-3f31-4b94-b756-2515f517a886 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:48:06.905628 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:48:06.929614 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8202d04-9ea7-4a02-a404-e39bc0097b95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:48:06.930698 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8202d04-9ea7-4a02-a404-e39bc0097b95 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:48:06.931177 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8202d04-9ea7-4a02-a404-e39bc0097b95 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:48:06.931606 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8202d04-9ea7-4a02-a404-e39bc0097b95 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:48:06.943606 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8202d04-9ea7-4a02-a404-e39bc0097b95 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:48:06.946245 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8202d04-9ea7-4a02-a404-e39bc0097b95 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:48:06.966992 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application [None req-c8202d04-9ea7-4a02-a404-e39bc0097b95 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 11:48:06.966992 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:48:06.966992 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:48:06.966992 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:48:06.966992 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:48:06.966992 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:48:06.966992 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:48:06.966992 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:48:06.966992 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:48:06.966992 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:48:06.966992 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:48:06.966992 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:48:06.966992 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 11 11:48:06.966992 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 11 11:48:06.966992 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 11 11:48:06.968082 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 11 11:48:06.968082 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:48:06.968082 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:48:06.968082 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:48:06.968082 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:48:06.968082 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:48:06.968082 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:48:06.968082 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:48:06.968082 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application with Lock( Oct 11 11:48:06.968082 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:48:06.968082 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:48:06.968082 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:48:06.968082 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:48:06.968082 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:48:06.968082 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:48:06.968082 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:48:06.968082 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:48:06.969201 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 11 11:48:06.969201 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 11 11:48:06.969201 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 11 11:48:06.969201 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 11 11:48:06.969201 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 11 11:48:06.969201 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 11 11:48:06.969201 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 11:48:06.969201 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:48:06.970060 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 419/829] 10.222.0.59 () {56 vars in 1111 bytes} [Tue Oct 11 11:48:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:48:06.975824 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9977953-98bb-4744-9ace-3a2e8de18aa1 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:48:07.078356 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9977953-98bb-4744-9ace-3a2e8de18aa1 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:48:07.080010 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:48:07.118229 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae55ba1a-555c-44e6-a6de-99de913ff08a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:48:07.118879 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae55ba1a-555c-44e6-a6de-99de913ff08a None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:48:07.119150 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae55ba1a-555c-44e6-a6de-99de913ff08a None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:48:07.119427 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae55ba1a-555c-44e6-a6de-99de913ff08a None admin] PATH_INFO: `/v3/projects` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:48:07.120124 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae55ba1a-555c-44e6-a6de-99de913ff08a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:48:07.120422 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae55ba1a-555c-44e6-a6de-99de913ff08a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:48:07.121828 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae55ba1a-555c-44e6-a6de-99de913ff08a None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:48:07.134537 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 411/830] 10.222.0.59 () {56 vars in 1148 bytes} [Tue Oct 11 11:48:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 11:48:07.140948 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8202d04-9ea7-4a02-a404-e39bc0097b95 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:48:07.271276 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8202d04-9ea7-4a02-a404-e39bc0097b95 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:48:07.310403 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f314aad7-aec7-41eb-8313-6bca4d42364d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:48:07.311091 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f314aad7-aec7-41eb-8313-6bca4d42364d None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:48:07.311375 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f314aad7-aec7-41eb-8313-6bca4d42364d None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:48:07.311673 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f314aad7-aec7-41eb-8313-6bca4d42364d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:48:07.312538 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f314aad7-aec7-41eb-8313-6bca4d42364d None admin] RBAC: Adding query filter params (role.id=6a95127666ae4707bc09aca24e0d3d39, scope.project.id=26ecf552cc9a4e5ca1d380509683dc4a, user.id=bad5e1cb306b46818096940acc3810b2) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:48:07.312852 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f314aad7-aec7-41eb-8313-6bca4d42364d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:48:07.314357 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f314aad7-aec7-41eb-8313-6bca4d42364d None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:48:07.321155 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 420/831] 10.222.0.59 () {56 vars in 1374 bytes} [Tue Oct 11 11:48:07 2022] GET /identity/v3/role_assignments?user.id=bad5e1cb306b46818096940acc3810b2&scope.project.id=26ecf552cc9a4e5ca1d380509683dc4a&role.id=6a95127666ae4707bc09aca24e0d3d39 => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:48:08.586272 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a932cd5-02b5-4e2c-a19d-dbabef4f25ee None None] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:48:08.587012 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a932cd5-02b5-4e2c-a19d-dbabef4f25ee None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:48:08.587012 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a932cd5-02b5-4e2c-a19d-dbabef4f25ee None None] PATH_INFO: `/` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:48:08.588223 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 412/832] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:48:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:48:08.593723 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deff861-afbb-4921-8e7d-a9e5baf898ad None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:48:08.594188 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deff861-afbb-4921-8e7d-a9e5baf898ad None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:48:08.594301 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deff861-afbb-4921-8e7d-a9e5baf898ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:48:08.613581 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-8deff861-afbb-4921-8e7d-a9e5baf898ad None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:48:08.614775 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-8deff861-afbb-4921-8e7d-a9e5baf898ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:48:08.659271 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:48:08.691652 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 421/833] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:48:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 11 11:48:08.962975 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a932cd5-02b5-4e2c-a19d-dbabef4f25ee None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:48:09.070920 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a932cd5-02b5-4e2c-a19d-dbabef4f25ee None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:48:09.105865 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9fd542-a4ba-486a-9dd1-772af4c647df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:48:09.106482 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9fd542-a4ba-486a-9dd1-772af4c647df None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:48:09.106763 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9fd542-a4ba-486a-9dd1-772af4c647df None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:48:09.107038 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9fd542-a4ba-486a-9dd1-772af4c647df None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:48:09.117938 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9fd542-a4ba-486a-9dd1-772af4c647df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 11:48:09.122737 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application [None req-8d9fd542-a4ba-486a-9dd1-772af4c647df None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 11 11:48:09.122737 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:48:09.122737 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:48:09.122737 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:48:09.122737 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:48:09.122737 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:48:09.122737 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:48:09.122737 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:48:09.122737 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:48:09.122737 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:48:09.122737 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:48:09.122737 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:48:09.122737 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 11:48:09.122737 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 11:48:09.122737 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 11:48:09.122737 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 11:48:09.123809 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 11:48:09.123809 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 11:48:09.123809 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 11:48:09.123809 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 11:48:09.123809 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:48:09.123809 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:48:09.123809 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:48:09.123809 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:48:09.123809 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:48:09.123809 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:48:09.123809 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:48:09.123809 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application with Lock( Oct 11 11:48:09.123809 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:48:09.123809 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:48:09.123809 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:48:09.123809 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:48:09.124643 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:48:09.124643 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:48:09.124643 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:48:09.124643 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:48:09.124643 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 11:48:09.124643 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 11:48:09.124643 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 11:48:09.124643 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 11:48:09.124643 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 11:48:09.124643 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 11:48:09.124643 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 11 11:48:09.124643 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:48:09.124643 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 413/834] 10.222.0.59 () {56 vars in 1096 bytes} [Tue Oct 11 11:48:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:48:09.130238 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8deff861-afbb-4921-8e7d-a9e5baf898ad None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:48:09.233010 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8deff861-afbb-4921-8e7d-a9e5baf898ad None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:48:09.268385 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ffc0ce-d34b-471f-b803-91e0b7a9998e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:48:09.268983 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ffc0ce-d34b-471f-b803-91e0b7a9998e None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:48:09.269259 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ffc0ce-d34b-471f-b803-91e0b7a9998e None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:48:09.269530 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ffc0ce-d34b-471f-b803-91e0b7a9998e None admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:48:09.270209 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ffc0ce-d34b-471f-b803-91e0b7a9998e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:48:09.270514 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ffc0ce-d34b-471f-b803-91e0b7a9998e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:48:09.271928 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ffc0ce-d34b-471f-b803-91e0b7a9998e None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:48:09.280109 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 422/835] 10.222.0.59 () {56 vars in 1099 bytes} [Tue Oct 11 11:48:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:48:09.286761 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9fd542-a4ba-486a-9dd1-772af4c647df None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:48:09.312795 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:48:09.388669 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9fd542-a4ba-486a-9dd1-772af4c647df None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:48:09.422760 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1656d55-bbe1-45a0-988b-34902b16c58b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:48:09.423377 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1656d55-bbe1-45a0-988b-34902b16c58b None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:48:09.423651 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1656d55-bbe1-45a0-988b-34902b16c58b None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:48:09.423910 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1656d55-bbe1-45a0-988b-34902b16c58b None admin] PATH_INFO: `/v3/users/a133b2bca86240d0b751c42694520e12` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:48:09.425106 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1656d55-bbe1-45a0-988b-34902b16c58b None admin] RBAC: Authorizing `identity:get_user(user_id=a133b2bca86240d0b751c42694520e12)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:48:09.426539 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1656d55-bbe1-45a0-988b-34902b16c58b None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:48:09.428126 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 414/836] 10.222.0.59 () {56 vars in 1177 bytes} [Tue Oct 11 11:48:09 2022] GET /identity/v3/users/a133b2bca86240d0b751c42694520e12 => generated 285 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:48:09.434187 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ffc0ce-d34b-471f-b803-91e0b7a9998e None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:48:09.470819 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:48:09.537535 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ffc0ce-d34b-471f-b803-91e0b7a9998e None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:48:09.573184 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61406a96-a495-4c22-b862-f35a7b315831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:48:09.573841 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61406a96-a495-4c22-b862-f35a7b315831 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:48:09.574121 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61406a96-a495-4c22-b862-f35a7b315831 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:48:09.574388 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61406a96-a495-4c22-b862-f35a7b315831 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:48:09.643394 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61406a96-a495-4c22-b862-f35a7b315831 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:48:09.644851 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61406a96-a495-4c22-b862-f35a7b315831 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:48:09.679278 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:48:09.744635 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 423/837] 10.222.0.59 () {58 vars in 1282 bytes} [Tue Oct 11 11:48:09 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 11 11:48:09.751077 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1656d55-bbe1-45a0-988b-34902b16c58b None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:48:09.844720 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:48:09.854568 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1656d55-bbe1-45a0-988b-34902b16c58b None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:48:09.890361 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b2790b-ce19-450a-bacd-18010e437880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:48:09.891030 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b2790b-ce19-450a-bacd-18010e437880 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:48:09.891308 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b2790b-ce19-450a-bacd-18010e437880 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:48:09.891574 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b2790b-ce19-450a-bacd-18010e437880 None admin] PATH_INFO: `/v3/projects/f4196b8338fa43c1bd146eeef8bd2995` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:48:09.892743 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b2790b-ce19-450a-bacd-18010e437880 None admin] RBAC: Authorizing `identity:get_project(project_id=f4196b8338fa43c1bd146eeef8bd2995)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:48:09.894174 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b2790b-ce19-450a-bacd-18010e437880 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:48:09.895840 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 415/838] 10.222.0.59 () {56 vars in 1186 bytes} [Tue Oct 11 11:48:09 2022] GET /identity/v3/projects/f4196b8338fa43c1bd146eeef8bd2995 => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:48:09.902988 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61406a96-a495-4c22-b862-f35a7b315831 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:48:10.009283 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61406a96-a495-4c22-b862-f35a7b315831 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:48:10.048487 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b11e3e-ae66-456a-b527-995da1b76c0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:48:10.049117 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b11e3e-ae66-456a-b527-995da1b76c0b None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:48:10.049394 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b11e3e-ae66-456a-b527-995da1b76c0b None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:48:10.049696 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b11e3e-ae66-456a-b527-995da1b76c0b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:48:10.050406 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b11e3e-ae66-456a-b527-995da1b76c0b None admin] RBAC: Adding query filter params (role.id=5cf1f4bd2fef46ab9c6eff56d52f9f69, scope.project.id=f4196b8338fa43c1bd146eeef8bd2995, user.id=a133b2bca86240d0b751c42694520e12) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:48:10.050755 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b11e3e-ae66-456a-b527-995da1b76c0b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:48:10.052238 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b11e3e-ae66-456a-b527-995da1b76c0b None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:48:10.055696 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:48:10.059612 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 424/839] 10.222.0.59 () {56 vars in 1374 bytes} [Tue Oct 11 11:48:09 2022] GET /identity/v3/role_assignments?user.id=a133b2bca86240d0b751c42694520e12&scope.project.id=f4196b8338fa43c1bd146eeef8bd2995&role.id=5cf1f4bd2fef46ab9c6eff56d52f9f69 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:48:12.224970 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d788eff-74f1-4ac7-bef5-70f73a54c123 None None] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:48:12.225512 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d788eff-74f1-4ac7-bef5-70f73a54c123 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:48:12.225512 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d788eff-74f1-4ac7-bef5-70f73a54c123 None None] PATH_INFO: `/` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:48:12.226886 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 416/840] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:48:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:48:12.232483 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76a50bc-db00-441e-8d20-6252508a19e2 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:48:12.232885 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76a50bc-db00-441e-8d20-6252508a19e2 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:48:12.233024 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76a50bc-db00-441e-8d20-6252508a19e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:48:12.252424 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-f76a50bc-db00-441e-8d20-6252508a19e2 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:48:12.253596 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-f76a50bc-db00-441e-8d20-6252508a19e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:48:12.326153 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 425/841] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:48:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 11:48:12.590876 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d788eff-74f1-4ac7-bef5-70f73a54c123 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:48:12.698602 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d788eff-74f1-4ac7-bef5-70f73a54c123 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:48:12.733793 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1172440-28e8-4791-af7f-dd8cbed2f81b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:48:12.734431 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1172440-28e8-4791-af7f-dd8cbed2f81b None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:48:12.734768 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1172440-28e8-4791-af7f-dd8cbed2f81b None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:48:12.735038 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1172440-28e8-4791-af7f-dd8cbed2f81b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:48:12.736217 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1172440-28e8-4791-af7f-dd8cbed2f81b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:48:12.737638 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1172440-28e8-4791-af7f-dd8cbed2f81b None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:48:12.739238 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 417/842] 10.222.0.59 () {56 vars in 1108 bytes} [Tue Oct 11 11:48:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:48:12.745445 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76a50bc-db00-441e-8d20-6252508a19e2 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:48:12.850497 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76a50bc-db00-441e-8d20-6252508a19e2 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:48:12.879761 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:48:12.887086 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ff2884-2327-4053-9d0c-6e5d777661a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:48:12.887717 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ff2884-2327-4053-9d0c-6e5d777661a4 None admin] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:48:12.888012 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ff2884-2327-4053-9d0c-6e5d777661a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:48:12.888272 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ff2884-2327-4053-9d0c-6e5d777661a4 None admin] PATH_INFO: `/v3/users` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:48:12.888932 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ff2884-2327-4053-9d0c-6e5d777661a4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:48:12.890486 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ff2884-2327-4053-9d0c-6e5d777661a4 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:48:12.913430 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 426/843] 10.222.0.59 () {60 vars in 1132 bytes} [Tue Oct 11 11:48:12 2022] POST /identity/v3/users => generated 286 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 11:48:15.058778 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869f92b0-7118-4ee4-8be4-5f0a973edf76 None None] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:48:15.059401 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869f92b0-7118-4ee4-8be4-5f0a973edf76 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:48:15.059401 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869f92b0-7118-4ee4-8be4-5f0a973edf76 None None] PATH_INFO: `/` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:48:15.060667 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 418/844] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:48:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:48:15.068787 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3d50ad-2aca-4b52-b648-7b11cff35355 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:48:15.069275 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3d50ad-2aca-4b52-b648-7b11cff35355 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:48:15.069275 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3d50ad-2aca-4b52-b648-7b11cff35355 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:48:15.089030 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-2e3d50ad-2aca-4b52-b648-7b11cff35355 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:48:15.090214 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-2e3d50ad-2aca-4b52-b648-7b11cff35355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:48:15.160569 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 427/845] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:48:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 11:48:15.430741 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869f92b0-7118-4ee4-8be4-5f0a973edf76 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:48:15.459271 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:48:15.536602 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869f92b0-7118-4ee4-8be4-5f0a973edf76 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:48:15.571420 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260f6afb-5d59-4d61-b0d7-83e837ad06d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:48:15.572096 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260f6afb-5d59-4d61-b0d7-83e837ad06d7 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:48:15.572381 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260f6afb-5d59-4d61-b0d7-83e837ad06d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:48:15.572656 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260f6afb-5d59-4d61-b0d7-83e837ad06d7 None admin] PATH_INFO: `/v3/roles/3a1a5476791547419a0b885c2022ef68` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:48:15.574131 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260f6afb-5d59-4d61-b0d7-83e837ad06d7 None admin] RBAC: Authorizing `identity:get_role(role_id=3a1a5476791547419a0b885c2022ef68)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:48:15.575575 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260f6afb-5d59-4d61-b0d7-83e837ad06d7 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:48:15.577015 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 419/846] 10.222.0.59 () {56 vars in 1177 bytes} [Tue Oct 11 11:48:15 2022] GET /identity/v3/roles/3a1a5476791547419a0b885c2022ef68 => generated 223 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:48:15.583851 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3d50ad-2aca-4b52-b648-7b11cff35355 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:48:15.684035 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3d50ad-2aca-4b52-b648-7b11cff35355 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:48:15.701748 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:48:15.721272 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da25dde2-ac1c-4a3e-ac0b-039a47305920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:48:15.721970 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da25dde2-ac1c-4a3e-ac0b-039a47305920 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:48:15.722266 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da25dde2-ac1c-4a3e-ac0b-039a47305920 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:48:15.722545 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da25dde2-ac1c-4a3e-ac0b-039a47305920 None admin] PATH_INFO: `/v3/users/cf6b99878c1646f3b62abd8e1a5a5264` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:48:15.737249 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da25dde2-ac1c-4a3e-ac0b-039a47305920 None admin] RBAC: Authorizing `identity:get_user(user_id=cf6b99878c1646f3b62abd8e1a5a5264)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:48:15.738864 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da25dde2-ac1c-4a3e-ac0b-039a47305920 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:48:15.740523 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 428/847] 10.222.0.59 () {56 vars in 1177 bytes} [Tue Oct 11 11:48:15 2022] GET /identity/v3/users/cf6b99878c1646f3b62abd8e1a5a5264 => generated 286 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:48:15.746827 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260f6afb-5d59-4d61-b0d7-83e837ad06d7 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:48:15.845804 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:48:15.853041 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260f6afb-5d59-4d61-b0d7-83e837ad06d7 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:48:15.888763 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0f086f-e7fb-4719-afbb-f84ec67b9693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:48:15.889385 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0f086f-e7fb-4719-afbb-f84ec67b9693 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:48:15.889659 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0f086f-e7fb-4719-afbb-f84ec67b9693 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:48:15.889923 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0f086f-e7fb-4719-afbb-f84ec67b9693 None admin] PATH_INFO: `/v3/projects/f4196b8338fa43c1bd146eeef8bd2995` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:48:15.891278 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0f086f-e7fb-4719-afbb-f84ec67b9693 None admin] RBAC: Authorizing `identity:get_project(project_id=f4196b8338fa43c1bd146eeef8bd2995)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:48:15.892815 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0f086f-e7fb-4719-afbb-f84ec67b9693 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:48:15.894683 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 420/848] 10.222.0.59 () {56 vars in 1186 bytes} [Tue Oct 11 11:48:15 2022] GET /identity/v3/projects/f4196b8338fa43c1bd146eeef8bd2995 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:48:15.901156 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da25dde2-ac1c-4a3e-ac0b-039a47305920 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:48:16.004208 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da25dde2-ac1c-4a3e-ac0b-039a47305920 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:48:16.039813 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd54321-d67b-481a-b4ad-a9c34714f009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:48:16.040456 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd54321-d67b-481a-b4ad-a9c34714f009 None admin] REQUEST_METHOD: `PUT` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:48:16.040729 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd54321-d67b-481a-b4ad-a9c34714f009 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:48:16.040996 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd54321-d67b-481a-b4ad-a9c34714f009 None admin] PATH_INFO: `/v3/projects/f4196b8338fa43c1bd146eeef8bd2995/users/cf6b99878c1646f3b62abd8e1a5a5264/roles/3a1a5476791547419a0b885c2022ef68` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:48:16.043494 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd54321-d67b-481a-b4ad-a9c34714f009 None admin] RBAC: Authorizing `identity:create_grant(project_id=f4196b8338fa43c1bd146eeef8bd2995, user_id=cf6b99878c1646f3b62abd8e1a5a5264, role_id=3a1a5476791547419a0b885c2022ef68)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:48:16.044952 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd54321-d67b-481a-b4ad-a9c34714f009 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:48:16.059407 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 429/849] 10.222.0.59 () {58 vars in 1439 bytes} [Tue Oct 11 11:48:15 2022] PUT /identity/v3/projects/f4196b8338fa43c1bd146eeef8bd2995/users/cf6b99878c1646f3b62abd8e1a5a5264/roles/3a1a5476791547419a0b885c2022ef68 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 11:48:18.199601 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4341f5f-525e-494f-82e2-cd79f2563be5 None None] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:48:18.200188 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4341f5f-525e-494f-82e2-cd79f2563be5 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:48:18.200188 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4341f5f-525e-494f-82e2-cd79f2563be5 None None] PATH_INFO: `/` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:48:18.201574 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 421/850] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:48:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:48:18.207588 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71437a7-2b55-46b7-94b0-b9dd378f6afc None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:48:18.208087 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71437a7-2b55-46b7-94b0-b9dd378f6afc None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:48:18.208087 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71437a7-2b55-46b7-94b0-b9dd378f6afc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:48:18.229998 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-f71437a7-2b55-46b7-94b0-b9dd378f6afc None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:48:18.231150 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-f71437a7-2b55-46b7-94b0-b9dd378f6afc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:48:18.262110 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:48:18.308416 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 430/851] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:48:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 11:48:18.573818 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4341f5f-525e-494f-82e2-cd79f2563be5 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:48:18.683131 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4341f5f-525e-494f-82e2-cd79f2563be5 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:48:18.718494 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399c2f4b-9787-4fb7-8dea-bdca9dbf9aa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:48:18.719149 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399c2f4b-9787-4fb7-8dea-bdca9dbf9aa9 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:48:18.719425 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399c2f4b-9787-4fb7-8dea-bdca9dbf9aa9 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:48:18.719724 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399c2f4b-9787-4fb7-8dea-bdca9dbf9aa9 None admin] PATH_INFO: `/v3/roles/3a1a5476791547419a0b885c2022ef68` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:48:18.721153 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399c2f4b-9787-4fb7-8dea-bdca9dbf9aa9 None admin] RBAC: Authorizing `identity:get_role(role_id=3a1a5476791547419a0b885c2022ef68)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:48:18.722512 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399c2f4b-9787-4fb7-8dea-bdca9dbf9aa9 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:48:18.724176 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 422/852] 10.222.0.59 () {56 vars in 1177 bytes} [Tue Oct 11 11:48:18 2022] GET /identity/v3/roles/3a1a5476791547419a0b885c2022ef68 => generated 223 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:48:18.730291 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71437a7-2b55-46b7-94b0-b9dd378f6afc None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:48:18.831380 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71437a7-2b55-46b7-94b0-b9dd378f6afc None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:48:18.867100 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb63753d-eeeb-4aee-9fd2-930fee6424f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:48:18.867769 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb63753d-eeeb-4aee-9fd2-930fee6424f9 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:48:18.868047 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb63753d-eeeb-4aee-9fd2-930fee6424f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:48:18.868327 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb63753d-eeeb-4aee-9fd2-930fee6424f9 None admin] PATH_INFO: `/v3/users/cf6b99878c1646f3b62abd8e1a5a5264` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:48:18.869505 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb63753d-eeeb-4aee-9fd2-930fee6424f9 None admin] RBAC: Authorizing `identity:get_user(user_id=cf6b99878c1646f3b62abd8e1a5a5264)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:48:18.871016 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb63753d-eeeb-4aee-9fd2-930fee6424f9 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:48:18.872606 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 431/853] 10.222.0.59 () {56 vars in 1177 bytes} [Tue Oct 11 11:48:18 2022] GET /identity/v3/users/cf6b99878c1646f3b62abd8e1a5a5264 => generated 286 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:48:18.878926 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399c2f4b-9787-4fb7-8dea-bdca9dbf9aa9 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:48:18.911825 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:48:18.987228 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399c2f4b-9787-4fb7-8dea-bdca9dbf9aa9 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:48:19.023552 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8943446-aee3-4c99-b349-efb1b6d05c7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:48:19.024150 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8943446-aee3-4c99-b349-efb1b6d05c7d None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:48:19.024433 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8943446-aee3-4c99-b349-efb1b6d05c7d None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:48:19.024705 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8943446-aee3-4c99-b349-efb1b6d05c7d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:48:19.096757 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8943446-aee3-4c99-b349-efb1b6d05c7d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:48:19.098116 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8943446-aee3-4c99-b349-efb1b6d05c7d None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:48:19.128540 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:48:19.200007 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 423/854] 10.222.0.59 () {58 vars in 1282 bytes} [Tue Oct 11 11:48:18 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 11 11:48:19.206438 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb63753d-eeeb-4aee-9fd2-930fee6424f9 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:48:19.238602 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:48:19.309861 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb63753d-eeeb-4aee-9fd2-930fee6424f9 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:48:19.344950 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2e94a0-6433-42db-b193-afdaace14b2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:48:19.345567 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2e94a0-6433-42db-b193-afdaace14b2e None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:48:19.345845 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2e94a0-6433-42db-b193-afdaace14b2e None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:48:19.346093 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2e94a0-6433-42db-b193-afdaace14b2e None admin] PATH_INFO: `/v3/projects/f4196b8338fa43c1bd146eeef8bd2995` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:48:19.347347 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2e94a0-6433-42db-b193-afdaace14b2e None admin] RBAC: Authorizing `identity:get_project(project_id=f4196b8338fa43c1bd146eeef8bd2995)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:48:19.348722 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2e94a0-6433-42db-b193-afdaace14b2e None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:48:19.350349 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 432/855] 10.222.0.59 () {56 vars in 1186 bytes} [Tue Oct 11 11:48:19 2022] GET /identity/v3/projects/f4196b8338fa43c1bd146eeef8bd2995 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:48:19.356921 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8943446-aee3-4c99-b349-efb1b6d05c7d None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:48:19.458055 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8943446-aee3-4c99-b349-efb1b6d05c7d None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:48:19.487011 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:48:19.494071 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89858cc-fe39-47f2-9168-795f75682761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:48:19.494743 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89858cc-fe39-47f2-9168-795f75682761 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:48:19.495018 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89858cc-fe39-47f2-9168-795f75682761 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:48:19.495289 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89858cc-fe39-47f2-9168-795f75682761 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:48:19.496009 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89858cc-fe39-47f2-9168-795f75682761 None admin] RBAC: Adding query filter params (role.id=3a1a5476791547419a0b885c2022ef68, scope.project.id=f4196b8338fa43c1bd146eeef8bd2995, user.id=cf6b99878c1646f3b62abd8e1a5a5264) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:48:19.496309 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89858cc-fe39-47f2-9168-795f75682761 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:48:19.497739 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89858cc-fe39-47f2-9168-795f75682761 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:48:19.504041 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 424/856] 10.222.0.59 () {56 vars in 1374 bytes} [Tue Oct 11 11:48:19 2022] GET /identity/v3/role_assignments?user.id=cf6b99878c1646f3b62abd8e1a5a5264&scope.project.id=f4196b8338fa43c1bd146eeef8bd2995&role.id=3a1a5476791547419a0b885c2022ef68 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:48:21.618925 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9f296e-462e-4b0b-bee7-142b19e176e3 None None] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:48:21.619515 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9f296e-462e-4b0b-bee7-142b19e176e3 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:48:21.619515 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9f296e-462e-4b0b-bee7-142b19e176e3 None None] PATH_INFO: `/` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:48:21.620815 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 433/857] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:48:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:48:21.626504 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698597d4-4eba-4531-ab51-ae00e3bc8e9c None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:48:21.626974 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698597d4-4eba-4531-ab51-ae00e3bc8e9c None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:48:21.627074 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698597d4-4eba-4531-ab51-ae00e3bc8e9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:48:21.645866 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-698597d4-4eba-4531-ab51-ae00e3bc8e9c None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:48:21.647083 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-698597d4-4eba-4531-ab51-ae00e3bc8e9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:48:21.714718 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 425/858] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:48:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 11 11:48:21.982672 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9f296e-462e-4b0b-bee7-142b19e176e3 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:48:22.082898 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:48:22.087344 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9f296e-462e-4b0b-bee7-142b19e176e3 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:48:22.122882 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4af482c-6986-4764-a00a-b78a27f36a47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:48:22.123500 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4af482c-6986-4764-a00a-b78a27f36a47 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:48:22.123781 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4af482c-6986-4764-a00a-b78a27f36a47 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:48:22.124044 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4af482c-6986-4764-a00a-b78a27f36a47 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:48:22.125246 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4af482c-6986-4764-a00a-b78a27f36a47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:48:22.126719 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4af482c-6986-4764-a00a-b78a27f36a47 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:48:22.128282 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 434/859] 10.222.0.59 () {56 vars in 1108 bytes} [Tue Oct 11 11:48:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:48:22.134379 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698597d4-4eba-4531-ab51-ae00e3bc8e9c None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:48:22.238520 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698597d4-4eba-4531-ab51-ae00e3bc8e9c None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:48:22.262750 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:48:22.275234 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e5e4b6-3ea6-49c3-9e1c-711390e78fdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:48:22.275855 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e5e4b6-3ea6-49c3-9e1c-711390e78fdb None admin] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:48:22.276134 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e5e4b6-3ea6-49c3-9e1c-711390e78fdb None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:48:22.276406 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e5e4b6-3ea6-49c3-9e1c-711390e78fdb None admin] PATH_INFO: `/v3/projects` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:48:22.277062 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e5e4b6-3ea6-49c3-9e1c-711390e78fdb None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:48:22.278496 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e5e4b6-3ea6-49c3-9e1c-711390e78fdb None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:48:22.306643 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 426/860] 10.222.0.59 () {60 vars in 1141 bytes} [Tue Oct 11 11:48:22 2022] POST /identity/v3/projects => generated 311 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 11:48:24.489442 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205f57eb-90c7-4cda-8fb5-9ccc449a30f0 None None] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:48:24.490050 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205f57eb-90c7-4cda-8fb5-9ccc449a30f0 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:48:24.490050 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205f57eb-90c7-4cda-8fb5-9ccc449a30f0 None None] PATH_INFO: `/` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:48:24.491404 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 435/861] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:48:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:48:24.497387 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6919163d-44a9-4975-8d16-0a30a807651f None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:48:24.497859 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6919163d-44a9-4975-8d16-0a30a807651f None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:48:24.497968 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6919163d-44a9-4975-8d16-0a30a807651f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:48:24.518453 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-6919163d-44a9-4975-8d16-0a30a807651f None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:48:24.519798 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-6919163d-44a9-4975-8d16-0a30a807651f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:48:24.593268 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 427/862] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:48:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 11:48:24.855192 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205f57eb-90c7-4cda-8fb5-9ccc449a30f0 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:48:24.962663 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205f57eb-90c7-4cda-8fb5-9ccc449a30f0 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:48:25.003045 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6ee240-e2ed-44dc-a4d6-6974faa77980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:48:25.003668 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6ee240-e2ed-44dc-a4d6-6974faa77980 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:48:25.003939 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6ee240-e2ed-44dc-a4d6-6974faa77980 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:48:25.004203 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6ee240-e2ed-44dc-a4d6-6974faa77980 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:48:25.005365 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6ee240-e2ed-44dc-a4d6-6974faa77980 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:48:25.006871 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6ee240-e2ed-44dc-a4d6-6974faa77980 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:48:25.008464 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 436/863] 10.222.0.59 () {56 vars in 1108 bytes} [Tue Oct 11 11:48:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:48:25.014585 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6919163d-44a9-4975-8d16-0a30a807651f None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:48:25.115918 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6919163d-44a9-4975-8d16-0a30a807651f None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:48:25.122996 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:48:25.155359 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01613db2-47d7-44e2-8f63-da9cb277f327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:48:25.156109 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01613db2-47d7-44e2-8f63-da9cb277f327 None admin] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:48:25.156387 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01613db2-47d7-44e2-8f63-da9cb277f327 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:48:25.156657 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01613db2-47d7-44e2-8f63-da9cb277f327 None admin] PATH_INFO: `/v3/users` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:48:25.157306 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01613db2-47d7-44e2-8f63-da9cb277f327 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:48:25.159134 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01613db2-47d7-44e2-8f63-da9cb277f327 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:48:25.181931 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 428/864] 10.222.0.59 () {60 vars in 1132 bytes} [Tue Oct 11 11:48:25 2022] POST /identity/v3/users => generated 286 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 11:48:27.387608 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba224f62-bbcc-4057-8385-598f503d29a1 None None] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:48:27.388258 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba224f62-bbcc-4057-8385-598f503d29a1 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:48:27.388258 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba224f62-bbcc-4057-8385-598f503d29a1 None None] PATH_INFO: `/` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:48:27.389618 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 437/865] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:48:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:48:27.395005 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab29eee-afb6-45b3-871c-2d3202bd1f9f None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:48:27.395478 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab29eee-afb6-45b3-871c-2d3202bd1f9f None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:48:27.395569 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab29eee-afb6-45b3-871c-2d3202bd1f9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:48:27.415002 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-fab29eee-afb6-45b3-871c-2d3202bd1f9f None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:48:27.416201 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-fab29eee-afb6-45b3-871c-2d3202bd1f9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:48:27.486192 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 429/866] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:48:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 11:48:27.760186 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba224f62-bbcc-4057-8385-598f503d29a1 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:48:27.801867 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:48:27.869129 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba224f62-bbcc-4057-8385-598f503d29a1 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:48:27.903421 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c2cc98-bb3b-42e4-bc5c-a2d84f7b950b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:48:27.904029 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c2cc98-bb3b-42e4-bc5c-a2d84f7b950b None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:48:27.904301 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c2cc98-bb3b-42e4-bc5c-a2d84f7b950b None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:48:27.904574 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c2cc98-bb3b-42e4-bc5c-a2d84f7b950b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:48:27.915180 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c2cc98-bb3b-42e4-bc5c-a2d84f7b950b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 11:48:27.919969 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application [None req-96c2cc98-bb3b-42e4-bc5c-a2d84f7b950b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 11 11:48:27.919969 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:48:27.919969 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:48:27.919969 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:48:27.919969 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:48:27.919969 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:48:27.919969 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:48:27.919969 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:48:27.919969 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:48:27.919969 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:48:27.919969 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:48:27.919969 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:48:27.919969 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 11:48:27.919969 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 11:48:27.919969 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 11:48:27.919969 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 11:48:27.921099 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 11:48:27.921099 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 11:48:27.921099 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 11:48:27.921099 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 11:48:27.921099 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:48:27.921099 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:48:27.921099 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:48:27.921099 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:48:27.921099 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:48:27.921099 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:48:27.921099 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:48:27.921099 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application with Lock( Oct 11 11:48:27.921099 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:48:27.921099 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:48:27.921099 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:48:27.921099 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:48:27.922116 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:48:27.922116 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:48:27.922116 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:48:27.922116 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:48:27.922116 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 11:48:27.922116 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 11:48:27.922116 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 11:48:27.922116 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 11:48:27.922116 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 11:48:27.922116 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 11:48:27.922116 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 11 11:48:27.922116 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:48:27.922116 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 438/867] 10.222.0.59 () {56 vars in 1096 bytes} [Tue Oct 11 11:48:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:48:27.927702 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab29eee-afb6-45b3-871c-2d3202bd1f9f None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:48:28.019626 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:48:28.029778 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab29eee-afb6-45b3-871c-2d3202bd1f9f None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:48:28.067557 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270481af-af94-4960-bf9c-685aa5d5f802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:48:28.068198 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270481af-af94-4960-bf9c-685aa5d5f802 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:48:28.068478 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270481af-af94-4960-bf9c-685aa5d5f802 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:48:28.068750 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270481af-af94-4960-bf9c-685aa5d5f802 None admin] PATH_INFO: `/v3/roles` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:48:28.069441 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270481af-af94-4960-bf9c-685aa5d5f802 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:48:28.069743 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270481af-af94-4960-bf9c-685aa5d5f802 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:48:28.071228 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270481af-af94-4960-bf9c-685aa5d5f802 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:48:28.079951 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 430/868] 10.222.0.59 () {56 vars in 1099 bytes} [Tue Oct 11 11:48:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:48:28.086364 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c2cc98-bb3b-42e4-bc5c-a2d84f7b950b None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:48:28.182186 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:48:28.189198 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c2cc98-bb3b-42e4-bc5c-a2d84f7b950b None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:48:28.226476 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be447328-4a5a-4e2e-b8dd-d787e6d71f6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:48:28.227148 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be447328-4a5a-4e2e-b8dd-d787e6d71f6d None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:48:28.227421 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be447328-4a5a-4e2e-b8dd-d787e6d71f6d None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:48:28.227688 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be447328-4a5a-4e2e-b8dd-d787e6d71f6d None admin] PATH_INFO: `/v3/users/03dab257def742a9b90af6d3e428ac9f` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:48:28.241602 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be447328-4a5a-4e2e-b8dd-d787e6d71f6d None admin] RBAC: Authorizing `identity:get_user(user_id=03dab257def742a9b90af6d3e428ac9f)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:48:28.243597 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be447328-4a5a-4e2e-b8dd-d787e6d71f6d None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:48:28.245259 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 439/869] 10.222.0.59 () {56 vars in 1177 bytes} [Tue Oct 11 11:48:28 2022] GET /identity/v3/users/03dab257def742a9b90af6d3e428ac9f => generated 286 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:48:28.251798 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270481af-af94-4960-bf9c-685aa5d5f802 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:48:28.353556 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270481af-af94-4960-bf9c-685aa5d5f802 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:48:28.387526 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d6614a-cdaf-43da-aa22-78d14b5978ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:48:28.388153 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d6614a-cdaf-43da-aa22-78d14b5978ba None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:48:28.388426 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d6614a-cdaf-43da-aa22-78d14b5978ba None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:48:28.388698 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d6614a-cdaf-43da-aa22-78d14b5978ba None admin] PATH_INFO: `/v3/projects/0a01178fe6a04ddfb6d7edf03d63eb48` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:48:28.389882 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d6614a-cdaf-43da-aa22-78d14b5978ba None admin] RBAC: Authorizing `identity:get_project(project_id=0a01178fe6a04ddfb6d7edf03d63eb48)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:48:28.391361 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d6614a-cdaf-43da-aa22-78d14b5978ba None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:48:28.392978 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 431/870] 10.222.0.59 () {56 vars in 1186 bytes} [Tue Oct 11 11:48:28 2022] GET /identity/v3/projects/0a01178fe6a04ddfb6d7edf03d63eb48 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:48:28.399310 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be447328-4a5a-4e2e-b8dd-d787e6d71f6d None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:48:28.501747 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be447328-4a5a-4e2e-b8dd-d787e6d71f6d None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:48:28.538812 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc4223b-b029-42da-a91f-61e1fa1dbb3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:48:28.539497 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc4223b-b029-42da-a91f-61e1fa1dbb3a None admin] REQUEST_METHOD: `PUT` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:48:28.539775 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc4223b-b029-42da-a91f-61e1fa1dbb3a None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:48:28.540059 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc4223b-b029-42da-a91f-61e1fa1dbb3a None admin] PATH_INFO: `/v3/projects/0a01178fe6a04ddfb6d7edf03d63eb48/users/03dab257def742a9b90af6d3e428ac9f/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:48:28.542081 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc4223b-b029-42da-a91f-61e1fa1dbb3a None admin] RBAC: Authorizing `identity:create_grant(project_id=0a01178fe6a04ddfb6d7edf03d63eb48, user_id=03dab257def742a9b90af6d3e428ac9f, role_id=5cf1f4bd2fef46ab9c6eff56d52f9f69)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:48:28.543570 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc4223b-b029-42da-a91f-61e1fa1dbb3a None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:48:28.558575 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 440/871] 10.222.0.59 () {58 vars in 1439 bytes} [Tue Oct 11 11:48:28 2022] PUT /identity/v3/projects/0a01178fe6a04ddfb6d7edf03d63eb48/users/03dab257def742a9b90af6d3e428ac9f/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 11:48:30.705510 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c896a9-5abd-4b4f-8520-6d589c60f0b5 None None] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:48:30.706187 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c896a9-5abd-4b4f-8520-6d589c60f0b5 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:48:30.706187 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c896a9-5abd-4b4f-8520-6d589c60f0b5 None None] PATH_INFO: `/` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:48:30.707437 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 432/872] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:48:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:48:30.712873 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee709d3-0b1a-4b18-8bc4-703fa6b3283d None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:48:30.713287 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee709d3-0b1a-4b18-8bc4-703fa6b3283d None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:48:30.713395 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee709d3-0b1a-4b18-8bc4-703fa6b3283d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:48:30.732814 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-2ee709d3-0b1a-4b18-8bc4-703fa6b3283d None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:48:30.734134 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-2ee709d3-0b1a-4b18-8bc4-703fa6b3283d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:48:30.744054 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:48:30.805110 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 441/873] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:48:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 11:48:31.069956 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c896a9-5abd-4b4f-8520-6d589c60f0b5 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:48:31.099673 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:48:31.183221 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c896a9-5abd-4b4f-8520-6d589c60f0b5 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:48:31.220658 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb96518b-2908-4616-9a0c-85568b3a3401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:48:31.221172 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb96518b-2908-4616-9a0c-85568b3a3401 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:48:31.221448 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb96518b-2908-4616-9a0c-85568b3a3401 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:48:31.221718 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb96518b-2908-4616-9a0c-85568b3a3401 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:48:31.232645 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb96518b-2908-4616-9a0c-85568b3a3401 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 11:48:31.237575 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application [None req-cb96518b-2908-4616-9a0c-85568b3a3401 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 11 11:48:31.237575 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:48:31.237575 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:48:31.237575 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:48:31.237575 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:48:31.237575 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:48:31.237575 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:48:31.237575 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:48:31.237575 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:48:31.237575 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:48:31.237575 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:48:31.237575 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:48:31.237575 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 11:48:31.237575 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 11:48:31.237575 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 11:48:31.237575 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 11:48:31.238712 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 11:48:31.238712 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 11:48:31.238712 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 11:48:31.238712 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 11:48:31.238712 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:48:31.238712 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:48:31.238712 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:48:31.238712 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:48:31.238712 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:48:31.238712 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:48:31.238712 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:48:31.238712 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application with Lock( Oct 11 11:48:31.238712 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:48:31.238712 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:48:31.238712 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:48:31.238712 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:48:31.239740 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:48:31.239740 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:48:31.239740 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:48:31.239740 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:48:31.239740 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 11:48:31.239740 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 11:48:31.239740 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 11:48:31.239740 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 11:48:31.239740 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 11:48:31.239740 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 11:48:31.239740 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 11 11:48:31.239740 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:48:31.239740 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 433/874] 10.222.0.59 () {56 vars in 1096 bytes} [Tue Oct 11 11:48:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:48:31.245302 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee709d3-0b1a-4b18-8bc4-703fa6b3283d None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:48:31.349035 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee709d3-0b1a-4b18-8bc4-703fa6b3283d None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:48:31.383935 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b7c279-2ae6-45d7-89d6-044a5495c3f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:48:31.384538 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b7c279-2ae6-45d7-89d6-044a5495c3f8 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:48:31.384830 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b7c279-2ae6-45d7-89d6-044a5495c3f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:48:31.385100 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b7c279-2ae6-45d7-89d6-044a5495c3f8 None admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:48:31.385786 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b7c279-2ae6-45d7-89d6-044a5495c3f8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:48:31.386093 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b7c279-2ae6-45d7-89d6-044a5495c3f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:48:31.387516 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b7c279-2ae6-45d7-89d6-044a5495c3f8 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:48:31.395947 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 442/875] 10.222.0.59 () {56 vars in 1099 bytes} [Tue Oct 11 11:48:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:48:31.402461 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb96518b-2908-4616-9a0c-85568b3a3401 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:48:31.491358 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:48:31.511375 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb96518b-2908-4616-9a0c-85568b3a3401 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:48:31.543627 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcb89d2-bf01-4f8b-8c4a-e83228c6e2fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:48:31.544245 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcb89d2-bf01-4f8b-8c4a-e83228c6e2fd None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:48:31.544517 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcb89d2-bf01-4f8b-8c4a-e83228c6e2fd None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:48:31.544782 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcb89d2-bf01-4f8b-8c4a-e83228c6e2fd None admin] PATH_INFO: `/v3/users/03dab257def742a9b90af6d3e428ac9f` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:48:31.545966 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcb89d2-bf01-4f8b-8c4a-e83228c6e2fd None admin] RBAC: Authorizing `identity:get_user(user_id=03dab257def742a9b90af6d3e428ac9f)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:48:31.547415 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcb89d2-bf01-4f8b-8c4a-e83228c6e2fd None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:48:31.548982 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 434/876] 10.222.0.59 () {56 vars in 1177 bytes} [Tue Oct 11 11:48:31 2022] GET /identity/v3/users/03dab257def742a9b90af6d3e428ac9f => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:48:31.555238 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b7c279-2ae6-45d7-89d6-044a5495c3f8 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:48:31.565457 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:48:31.653745 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b7c279-2ae6-45d7-89d6-044a5495c3f8 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:48:31.688038 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e927f6e0-9010-453e-8d9f-9ab694aeec71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:48:31.688631 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e927f6e0-9010-453e-8d9f-9ab694aeec71 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:48:31.688900 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e927f6e0-9010-453e-8d9f-9ab694aeec71 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:48:31.689166 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e927f6e0-9010-453e-8d9f-9ab694aeec71 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:48:31.756833 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e927f6e0-9010-453e-8d9f-9ab694aeec71 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:48:31.758222 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e927f6e0-9010-453e-8d9f-9ab694aeec71 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:48:31.766500 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:48:31.859029 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 443/877] 10.222.0.59 () {58 vars in 1282 bytes} [Tue Oct 11 11:48:31 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 11 11:48:31.865503 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcb89d2-bf01-4f8b-8c4a-e83228c6e2fd None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:48:31.969944 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcb89d2-bf01-4f8b-8c4a-e83228c6e2fd None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:48:32.005055 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d674e1-2733-4eca-9991-ba4270ff08a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:48:32.005688 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d674e1-2733-4eca-9991-ba4270ff08a8 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:48:32.005957 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d674e1-2733-4eca-9991-ba4270ff08a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:48:32.006220 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d674e1-2733-4eca-9991-ba4270ff08a8 None admin] PATH_INFO: `/v3/projects/0a01178fe6a04ddfb6d7edf03d63eb48` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:48:32.007430 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d674e1-2733-4eca-9991-ba4270ff08a8 None admin] RBAC: Authorizing `identity:get_project(project_id=0a01178fe6a04ddfb6d7edf03d63eb48)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:48:32.008774 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d674e1-2733-4eca-9991-ba4270ff08a8 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:48:32.010428 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 435/878] 10.222.0.59 () {56 vars in 1186 bytes} [Tue Oct 11 11:48:31 2022] GET /identity/v3/projects/0a01178fe6a04ddfb6d7edf03d63eb48 => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:48:32.017128 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e927f6e0-9010-453e-8d9f-9ab694aeec71 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:48:32.122368 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e927f6e0-9010-453e-8d9f-9ab694aeec71 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:48:32.129802 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:48:32.157512 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916073ca-5d44-4005-8ebb-abda00e42c20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:48:32.158101 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916073ca-5d44-4005-8ebb-abda00e42c20 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:48:32.158719 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916073ca-5d44-4005-8ebb-abda00e42c20 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:48:32.158986 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916073ca-5d44-4005-8ebb-abda00e42c20 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:48:32.159697 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916073ca-5d44-4005-8ebb-abda00e42c20 None admin] RBAC: Adding query filter params (role.id=5cf1f4bd2fef46ab9c6eff56d52f9f69, scope.project.id=0a01178fe6a04ddfb6d7edf03d63eb48, user.id=03dab257def742a9b90af6d3e428ac9f) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:48:32.160000 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916073ca-5d44-4005-8ebb-abda00e42c20 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:48:32.161427 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916073ca-5d44-4005-8ebb-abda00e42c20 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:48:32.167460 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 444/879] 10.222.0.59 () {56 vars in 1374 bytes} [Tue Oct 11 11:48:32 2022] GET /identity/v3/role_assignments?user.id=03dab257def742a9b90af6d3e428ac9f&scope.project.id=0a01178fe6a04ddfb6d7edf03d63eb48&role.id=5cf1f4bd2fef46ab9c6eff56d52f9f69 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:48:34.317439 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77687fd-9501-48c4-b1cf-38359fb087eb None None] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:48:34.317990 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77687fd-9501-48c4-b1cf-38359fb087eb None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:48:34.317990 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77687fd-9501-48c4-b1cf-38359fb087eb None None] PATH_INFO: `/` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:48:34.319342 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 436/880] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:48:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:48:34.325052 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34815ebd-ea04-49a9-a90d-f879275069d8 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:48:34.325473 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34815ebd-ea04-49a9-a90d-f879275069d8 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:48:34.325593 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34815ebd-ea04-49a9-a90d-f879275069d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:48:34.345455 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-34815ebd-ea04-49a9-a90d-f879275069d8 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:48:34.346705 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-34815ebd-ea04-49a9-a90d-f879275069d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:48:34.415546 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 445/881] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:48:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 11:48:34.679373 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77687fd-9501-48c4-b1cf-38359fb087eb None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:48:34.701764 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:48:34.790453 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77687fd-9501-48c4-b1cf-38359fb087eb None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:48:34.824832 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd90df2-3050-40c3-9887-205f8dc1043b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:48:34.825487 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd90df2-3050-40c3-9887-205f8dc1043b None admin] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:48:34.825762 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd90df2-3050-40c3-9887-205f8dc1043b None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:48:34.826024 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd90df2-3050-40c3-9887-205f8dc1043b None admin] PATH_INFO: `/v3/domains` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:48:34.826678 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd90df2-3050-40c3-9887-205f8dc1043b None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:48:34.828136 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd90df2-3050-40c3-9887-205f8dc1043b None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:48:34.844446 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 437/882] 10.222.0.59 () {60 vars in 1138 bytes} [Tue Oct 11 11:48:34 2022] POST /identity/v3/domains => generated 267 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 11:48:37.001472 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a67575-3f74-499e-a1c6-f289de51f289 None None] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:48:37.002054 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a67575-3f74-499e-a1c6-f289de51f289 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:48:37.002054 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a67575-3f74-499e-a1c6-f289de51f289 None None] PATH_INFO: `/` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:48:37.003433 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 446/883] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:48:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:48:37.009138 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5356998-7a2d-4542-b0e2-542c044b042a None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:48:37.009557 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5356998-7a2d-4542-b0e2-542c044b042a None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:48:37.009693 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5356998-7a2d-4542-b0e2-542c044b042a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:48:37.028978 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-a5356998-7a2d-4542-b0e2-542c044b042a None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:48:37.030164 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-a5356998-7a2d-4542-b0e2-542c044b042a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:48:37.102068 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 438/884] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:48:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 11:48:37.370515 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a67575-3f74-499e-a1c6-f289de51f289 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:48:37.474043 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:48:37.479077 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a67575-3f74-499e-a1c6-f289de51f289 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:48:37.514941 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb36e05f-b8dd-4b77-91ec-76f5c869572f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:48:37.515578 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb36e05f-b8dd-4b77-91ec-76f5c869572f None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:48:37.515866 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb36e05f-b8dd-4b77-91ec-76f5c869572f None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:48:37.516139 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb36e05f-b8dd-4b77-91ec-76f5c869572f None admin] PATH_INFO: `/v3/domains/ac7cf76efb1245daabddf824fadf9fb2` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:48:37.527707 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb36e05f-b8dd-4b77-91ec-76f5c869572f None admin] RBAC: Authorizing `identity:get_domain(domain_id=ac7cf76efb1245daabddf824fadf9fb2)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:48:37.529261 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb36e05f-b8dd-4b77-91ec-76f5c869572f None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:48:37.531055 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 447/885] 10.222.0.59 () {56 vars in 1183 bytes} [Tue Oct 11 11:48:37 2022] GET /identity/v3/domains/ac7cf76efb1245daabddf824fadf9fb2 => generated 267 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:48:37.537548 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5356998-7a2d-4542-b0e2-542c044b042a None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:48:37.548086 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:48:37.637986 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5356998-7a2d-4542-b0e2-542c044b042a None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:48:37.673785 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79e3224-8400-4b02-acb6-27f2ecbfb83d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:48:37.674382 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79e3224-8400-4b02-acb6-27f2ecbfb83d None admin] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:48:37.674660 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79e3224-8400-4b02-acb6-27f2ecbfb83d None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:48:37.674970 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79e3224-8400-4b02-acb6-27f2ecbfb83d None admin] PATH_INFO: `/v3/projects` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:48:37.675614 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79e3224-8400-4b02-acb6-27f2ecbfb83d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:48:37.677039 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79e3224-8400-4b02-acb6-27f2ecbfb83d None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:48:37.705562 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 439/886] 10.222.0.59 () {60 vars in 1141 bytes} [Tue Oct 11 11:48:37 2022] POST /identity/v3/projects => generated 361 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 11:48:39.854805 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c6bce9-f12e-4750-954a-bd3f1dd1c385 None None] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:48:39.855369 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c6bce9-f12e-4750-954a-bd3f1dd1c385 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:48:39.855369 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c6bce9-f12e-4750-954a-bd3f1dd1c385 None None] PATH_INFO: `/` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:48:39.856670 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 448/887] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:48:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:48:39.862140 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b714b3a-8cc5-4cdc-8ea8-9dadbf6a6c68 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:48:39.862626 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b714b3a-8cc5-4cdc-8ea8-9dadbf6a6c68 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:48:39.862722 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b714b3a-8cc5-4cdc-8ea8-9dadbf6a6c68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:48:39.882603 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-2b714b3a-8cc5-4cdc-8ea8-9dadbf6a6c68 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:48:39.883786 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-2b714b3a-8cc5-4cdc-8ea8-9dadbf6a6c68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:48:39.941083 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:48:39.954827 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 440/888] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:48:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 11:48:40.218953 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c6bce9-f12e-4750-954a-bd3f1dd1c385 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:48:40.328468 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c6bce9-f12e-4750-954a-bd3f1dd1c385 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:48:40.363405 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c973988-fadc-442f-991e-ee10bf214896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:48:40.364020 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c973988-fadc-442f-991e-ee10bf214896 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:48:40.364289 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c973988-fadc-442f-991e-ee10bf214896 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:48:40.364562 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c973988-fadc-442f-991e-ee10bf214896 None admin] PATH_INFO: `/v3/domains/ac7cf76efb1245daabddf824fadf9fb2` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:48:40.365788 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c973988-fadc-442f-991e-ee10bf214896 None admin] RBAC: Authorizing `identity:get_domain(domain_id=ac7cf76efb1245daabddf824fadf9fb2)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:48:40.367287 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c973988-fadc-442f-991e-ee10bf214896 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:48:40.368883 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 449/889] 10.222.0.59 () {56 vars in 1183 bytes} [Tue Oct 11 11:48:40 2022] GET /identity/v3/domains/ac7cf76efb1245daabddf824fadf9fb2 => generated 267 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:48:40.375126 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b714b3a-8cc5-4cdc-8ea8-9dadbf6a6c68 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:48:40.481443 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b714b3a-8cc5-4cdc-8ea8-9dadbf6a6c68 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:48:40.517045 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3384900-c69f-4ef5-8e0f-c66335c9bdf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:48:40.517650 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3384900-c69f-4ef5-8e0f-c66335c9bdf6 None admin] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:48:40.517927 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3384900-c69f-4ef5-8e0f-c66335c9bdf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:48:40.518201 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3384900-c69f-4ef5-8e0f-c66335c9bdf6 None admin] PATH_INFO: `/v3/users` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:48:40.518884 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3384900-c69f-4ef5-8e0f-c66335c9bdf6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:48:40.520408 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3384900-c69f-4ef5-8e0f-c66335c9bdf6 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:48:40.543739 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 441/890] 10.222.0.59 () {60 vars in 1132 bytes} [Tue Oct 11 11:48:40 2022] POST /identity/v3/users => generated 311 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 11:48:42.692481 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5f8a53-f234-4285-9d9a-378bc0e42776 None None] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:48:42.693077 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5f8a53-f234-4285-9d9a-378bc0e42776 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:48:42.693077 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5f8a53-f234-4285-9d9a-378bc0e42776 None None] PATH_INFO: `/` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:48:42.694360 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 450/891] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:48:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:48:42.700077 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872aa597-d826-43d7-a7fa-f483474ecb02 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:48:42.700512 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872aa597-d826-43d7-a7fa-f483474ecb02 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:48:42.700601 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872aa597-d826-43d7-a7fa-f483474ecb02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:48:42.720104 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-872aa597-d826-43d7-a7fa-f483474ecb02 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:48:42.721326 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-872aa597-d826-43d7-a7fa-f483474ecb02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:48:42.766716 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:48:42.794570 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 442/892] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:48:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 11:48:43.061374 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5f8a53-f234-4285-9d9a-378bc0e42776 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:48:43.092490 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:48:43.175062 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5f8a53-f234-4285-9d9a-378bc0e42776 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:48:43.212474 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6994ccdd-9a2b-4e3f-b90f-20e66f44ee28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:48:43.213144 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6994ccdd-9a2b-4e3f-b90f-20e66f44ee28 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:48:43.213484 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6994ccdd-9a2b-4e3f-b90f-20e66f44ee28 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:48:43.213703 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6994ccdd-9a2b-4e3f-b90f-20e66f44ee28 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:48:43.224946 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6994ccdd-9a2b-4e3f-b90f-20e66f44ee28 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 11:48:43.229811 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application [None req-6994ccdd-9a2b-4e3f-b90f-20e66f44ee28 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 11 11:48:43.229811 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:48:43.229811 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:48:43.229811 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:48:43.229811 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:48:43.229811 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:48:43.229811 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:48:43.229811 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:48:43.229811 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:48:43.229811 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:48:43.229811 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:48:43.229811 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:48:43.229811 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 11:48:43.229811 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 11:48:43.229811 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 11:48:43.229811 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 11:48:43.230908 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 11:48:43.230908 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 11:48:43.230908 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 11:48:43.230908 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 11:48:43.230908 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:48:43.230908 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:48:43.230908 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:48:43.230908 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:48:43.230908 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:48:43.230908 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:48:43.230908 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:48:43.230908 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application with Lock( Oct 11 11:48:43.230908 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:48:43.230908 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:48:43.230908 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:48:43.230908 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:48:43.232042 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:48:43.232042 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:48:43.232042 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:48:43.232042 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:48:43.232042 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 11:48:43.232042 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 11:48:43.232042 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 11:48:43.232042 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 11:48:43.232042 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 11:48:43.232042 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 11:48:43.232042 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 11 11:48:43.232042 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:48:43.232042 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 451/893] 10.222.0.59 () {56 vars in 1096 bytes} [Tue Oct 11 11:48:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:48:43.237561 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872aa597-d826-43d7-a7fa-f483474ecb02 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:48:43.344781 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872aa597-d826-43d7-a7fa-f483474ecb02 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:48:43.379183 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13b0d15-a4fc-4e65-a6d4-5e1bd8af5843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:48:43.379784 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13b0d15-a4fc-4e65-a6d4-5e1bd8af5843 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:48:43.380058 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13b0d15-a4fc-4e65-a6d4-5e1bd8af5843 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:48:43.380320 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13b0d15-a4fc-4e65-a6d4-5e1bd8af5843 None admin] PATH_INFO: `/v3/roles` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:48:43.380983 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13b0d15-a4fc-4e65-a6d4-5e1bd8af5843 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:48:43.381281 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13b0d15-a4fc-4e65-a6d4-5e1bd8af5843 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:48:43.382662 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13b0d15-a4fc-4e65-a6d4-5e1bd8af5843 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:48:43.390879 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 443/894] 10.222.0.59 () {56 vars in 1099 bytes} [Tue Oct 11 11:48:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:48:43.397006 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6994ccdd-9a2b-4e3f-b90f-20e66f44ee28 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:48:43.487121 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:48:43.499834 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6994ccdd-9a2b-4e3f-b90f-20e66f44ee28 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:48:43.535344 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df1fe73-149c-4759-b335-60c671bd9b87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:48:43.535977 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df1fe73-149c-4759-b335-60c671bd9b87 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:48:43.536250 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df1fe73-149c-4759-b335-60c671bd9b87 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:48:43.536515 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df1fe73-149c-4759-b335-60c671bd9b87 None admin] PATH_INFO: `/v3/users/9a83c99c0d9740a4addb9d14a19dd1de` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:48:43.549728 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df1fe73-149c-4759-b335-60c671bd9b87 None admin] RBAC: Authorizing `identity:get_user(user_id=9a83c99c0d9740a4addb9d14a19dd1de)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:48:43.551218 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df1fe73-149c-4759-b335-60c671bd9b87 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:48:43.552837 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 452/895] 10.222.0.59 () {56 vars in 1177 bytes} [Tue Oct 11 11:48:43 2022] GET /identity/v3/users/9a83c99c0d9740a4addb9d14a19dd1de => generated 311 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:48:43.558961 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13b0d15-a4fc-4e65-a6d4-5e1bd8af5843 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:48:43.602298 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:48:43.666264 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13b0d15-a4fc-4e65-a6d4-5e1bd8af5843 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:48:43.699353 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013030b5-44e2-4d02-9070-ec12422a6d9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:48:43.699978 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013030b5-44e2-4d02-9070-ec12422a6d9a None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:48:43.700253 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013030b5-44e2-4d02-9070-ec12422a6d9a None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:48:43.700841 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013030b5-44e2-4d02-9070-ec12422a6d9a None admin] PATH_INFO: `/v3/projects/39285d5b92fc49ac809128c5e7adeb18` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:48:43.701727 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013030b5-44e2-4d02-9070-ec12422a6d9a None admin] RBAC: Authorizing `identity:get_project(project_id=39285d5b92fc49ac809128c5e7adeb18)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:48:43.703186 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013030b5-44e2-4d02-9070-ec12422a6d9a None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:48:43.704813 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 444/896] 10.222.0.59 () {56 vars in 1186 bytes} [Tue Oct 11 11:48:43 2022] GET /identity/v3/projects/39285d5b92fc49ac809128c5e7adeb18 => generated 361 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:48:43.711049 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df1fe73-149c-4759-b335-60c671bd9b87 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:48:43.815614 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df1fe73-149c-4759-b335-60c671bd9b87 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:48:43.851595 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d166a0-5d80-44f4-96b1-641244d91642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:48:43.852261 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d166a0-5d80-44f4-96b1-641244d91642 None admin] REQUEST_METHOD: `PUT` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:48:43.852535 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d166a0-5d80-44f4-96b1-641244d91642 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:48:43.852803 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d166a0-5d80-44f4-96b1-641244d91642 None admin] PATH_INFO: `/v3/projects/39285d5b92fc49ac809128c5e7adeb18/users/9a83c99c0d9740a4addb9d14a19dd1de/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:48:43.855046 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d166a0-5d80-44f4-96b1-641244d91642 None admin] RBAC: Authorizing `identity:create_grant(project_id=39285d5b92fc49ac809128c5e7adeb18, user_id=9a83c99c0d9740a4addb9d14a19dd1de, role_id=5cf1f4bd2fef46ab9c6eff56d52f9f69)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:48:43.856501 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d166a0-5d80-44f4-96b1-641244d91642 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:48:43.871470 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 453/897] 10.222.0.59 () {58 vars in 1439 bytes} [Tue Oct 11 11:48:43 2022] PUT /identity/v3/projects/39285d5b92fc49ac809128c5e7adeb18/users/9a83c99c0d9740a4addb9d14a19dd1de/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 11:48:45.985837 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3063d88b-25cd-4b11-a321-2b169711459b None None] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:48:45.986442 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3063d88b-25cd-4b11-a321-2b169711459b None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:48:45.986442 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3063d88b-25cd-4b11-a321-2b169711459b None None] PATH_INFO: `/` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:48:45.987767 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 445/898] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:48:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:48:45.993410 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecb33a8-60df-4736-8b72-7db6d19552e2 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:48:45.993903 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecb33a8-60df-4736-8b72-7db6d19552e2 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:48:45.993903 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecb33a8-60df-4736-8b72-7db6d19552e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:48:46.007595 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:48:46.013860 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-1ecb33a8-60df-4736-8b72-7db6d19552e2 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:48:46.015833 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-1ecb33a8-60df-4736-8b72-7db6d19552e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:48:46.085649 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 454/899] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:48:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 11:48:46.349167 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3063d88b-25cd-4b11-a321-2b169711459b None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:48:46.456374 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3063d88b-25cd-4b11-a321-2b169711459b None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:48:46.464127 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:48:46.493916 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46a5430-cfa1-4553-b841-4b642c890b26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:48:46.494541 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46a5430-cfa1-4553-b841-4b642c890b26 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:48:46.494851 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46a5430-cfa1-4553-b841-4b642c890b26 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:48:46.495120 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46a5430-cfa1-4553-b841-4b642c890b26 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:48:46.505930 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46a5430-cfa1-4553-b841-4b642c890b26 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 11:48:46.510775 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application [None req-f46a5430-cfa1-4553-b841-4b642c890b26 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 11 11:48:46.510775 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:48:46.510775 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:48:46.510775 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:48:46.510775 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:48:46.510775 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:48:46.510775 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:48:46.510775 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:48:46.510775 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:48:46.510775 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:48:46.510775 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:48:46.510775 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:48:46.510775 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 11:48:46.510775 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 11:48:46.510775 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 11:48:46.510775 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 11:48:46.511914 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 11:48:46.511914 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 11:48:46.511914 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 11:48:46.511914 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 11:48:46.511914 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:48:46.511914 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:48:46.511914 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:48:46.511914 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:48:46.511914 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:48:46.511914 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:48:46.511914 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:48:46.511914 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application with Lock( Oct 11 11:48:46.511914 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:48:46.511914 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:48:46.511914 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:48:46.511914 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:48:46.513014 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:48:46.513014 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:48:46.513014 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:48:46.513014 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:48:46.513014 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 11:48:46.513014 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 11:48:46.513014 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 11:48:46.513014 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 11:48:46.513014 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 11:48:46.513014 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 11:48:46.513014 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 11 11:48:46.513014 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:48:46.513014 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 446/900] 10.222.0.59 () {56 vars in 1096 bytes} [Tue Oct 11 11:48:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:48:46.518436 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecb33a8-60df-4736-8b72-7db6d19552e2 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:48:46.624924 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecb33a8-60df-4736-8b72-7db6d19552e2 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:48:46.660130 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76553794-5453-494a-b015-ca36c3d27948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:48:46.660736 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76553794-5453-494a-b015-ca36c3d27948 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:48:46.661012 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76553794-5453-494a-b015-ca36c3d27948 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:48:46.661286 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76553794-5453-494a-b015-ca36c3d27948 None admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:48:46.661942 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76553794-5453-494a-b015-ca36c3d27948 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:48:46.662252 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76553794-5453-494a-b015-ca36c3d27948 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:48:46.663676 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76553794-5453-494a-b015-ca36c3d27948 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:48:46.665872 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:48:46.672433 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 455/901] 10.222.0.59 () {56 vars in 1099 bytes} [Tue Oct 11 11:48:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:48:46.678798 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46a5430-cfa1-4553-b841-4b642c890b26 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:48:46.781437 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46a5430-cfa1-4553-b841-4b642c890b26 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:48:46.817320 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f0eb9d-cb4d-43de-a365-cab165565de7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:48:46.817945 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f0eb9d-cb4d-43de-a365-cab165565de7 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:48:46.818221 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f0eb9d-cb4d-43de-a365-cab165565de7 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:48:46.818493 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f0eb9d-cb4d-43de-a365-cab165565de7 None admin] PATH_INFO: `/v3/users/9a83c99c0d9740a4addb9d14a19dd1de` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:48:46.819753 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f0eb9d-cb4d-43de-a365-cab165565de7 None admin] RBAC: Authorizing `identity:get_user(user_id=9a83c99c0d9740a4addb9d14a19dd1de)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:48:46.821180 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f0eb9d-cb4d-43de-a365-cab165565de7 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:48:46.822810 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 447/902] 10.222.0.59 () {56 vars in 1177 bytes} [Tue Oct 11 11:48:46 2022] GET /identity/v3/users/9a83c99c0d9740a4addb9d14a19dd1de => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:48:46.829187 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76553794-5453-494a-b015-ca36c3d27948 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:48:46.928746 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76553794-5453-494a-b015-ca36c3d27948 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:48:46.965277 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d4f314-5fee-4278-9531-2aabc769b4ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:48:46.965900 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d4f314-5fee-4278-9531-2aabc769b4ac None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:48:46.966184 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d4f314-5fee-4278-9531-2aabc769b4ac None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:48:46.966458 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d4f314-5fee-4278-9531-2aabc769b4ac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:48:47.032539 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:48:47.036634 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d4f314-5fee-4278-9531-2aabc769b4ac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:48:47.038143 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d4f314-5fee-4278-9531-2aabc769b4ac None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:48:47.139385 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 456/903] 10.222.0.59 () {58 vars in 1282 bytes} [Tue Oct 11 11:48:46 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 11 11:48:47.145848 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f0eb9d-cb4d-43de-a365-cab165565de7 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:48:47.182621 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:48:47.253572 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f0eb9d-cb4d-43de-a365-cab165565de7 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:48:47.289530 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3dc333-b174-4abc-a6dc-9b78d35dc202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:48:47.290163 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3dc333-b174-4abc-a6dc-9b78d35dc202 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:48:47.290446 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3dc333-b174-4abc-a6dc-9b78d35dc202 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:48:47.290749 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3dc333-b174-4abc-a6dc-9b78d35dc202 None admin] PATH_INFO: `/v3/projects/39285d5b92fc49ac809128c5e7adeb18` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:48:47.291906 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3dc333-b174-4abc-a6dc-9b78d35dc202 None admin] RBAC: Authorizing `identity:get_project(project_id=39285d5b92fc49ac809128c5e7adeb18)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:48:47.293352 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3dc333-b174-4abc-a6dc-9b78d35dc202 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:48:47.295089 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 448/904] 10.222.0.59 () {56 vars in 1186 bytes} [Tue Oct 11 11:48:47 2022] GET /identity/v3/projects/39285d5b92fc49ac809128c5e7adeb18 => generated 361 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:48:47.301599 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d4f314-5fee-4278-9531-2aabc769b4ac None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:48:47.407842 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:48:47.412255 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d4f314-5fee-4278-9531-2aabc769b4ac None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:48:47.447315 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c6d25c-f3aa-4db3-ac6f-fbfe1394d3e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:48:47.447829 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c6d25c-f3aa-4db3-ac6f-fbfe1394d3e1 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:48:47.448100 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c6d25c-f3aa-4db3-ac6f-fbfe1394d3e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:48:47.448366 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c6d25c-f3aa-4db3-ac6f-fbfe1394d3e1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:48:47.449084 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c6d25c-f3aa-4db3-ac6f-fbfe1394d3e1 None admin] RBAC: Adding query filter params (role.id=5cf1f4bd2fef46ab9c6eff56d52f9f69, scope.project.id=39285d5b92fc49ac809128c5e7adeb18, user.id=9a83c99c0d9740a4addb9d14a19dd1de) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:48:47.449388 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c6d25c-f3aa-4db3-ac6f-fbfe1394d3e1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:48:47.450865 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c6d25c-f3aa-4db3-ac6f-fbfe1394d3e1 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:48:47.456645 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 457/905] 10.222.0.59 () {56 vars in 1374 bytes} [Tue Oct 11 11:48:47 2022] GET /identity/v3/role_assignments?user.id=9a83c99c0d9740a4addb9d14a19dd1de&scope.project.id=39285d5b92fc49ac809128c5e7adeb18&role.id=5cf1f4bd2fef46ab9c6eff56d52f9f69 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:48:57.284601 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=49 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:48:57.454187 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=19 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:49:45.159195 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd06a6a-0264-4714-8881-e53a2d2dd3a7 None None] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:49:45.160341 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd06a6a-0264-4714-8881-e53a2d2dd3a7 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:49:45.160745 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd06a6a-0264-4714-8881-e53a2d2dd3a7 None None] PATH_INFO: `/` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:49:45.162511 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 449/906] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:49:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 11 11:49:45.168764 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb21cf9-fbd0-47da-9301-478b2a975f35 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:49:45.169545 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb21cf9-fbd0-47da-9301-478b2a975f35 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:49:45.169545 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb21cf9-fbd0-47da-9301-478b2a975f35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:49:45.192884 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-7cb21cf9-fbd0-47da-9301-478b2a975f35 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:49:45.194412 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-7cb21cf9-fbd0-47da-9301-478b2a975f35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:49:45.313974 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 458/907] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:49:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 11:49:45.635144 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd06a6a-0264-4714-8881-e53a2d2dd3a7 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:49:45.806991 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd06a6a-0264-4714-8881-e53a2d2dd3a7 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:49:45.873137 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d53bd36-2a18-40e1-9151-3fa52e647581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:49:45.874598 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d53bd36-2a18-40e1-9151-3fa52e647581 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:49:45.875126 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d53bd36-2a18-40e1-9151-3fa52e647581 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:49:45.875656 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d53bd36-2a18-40e1-9151-3fa52e647581 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:49:45.903574 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d53bd36-2a18-40e1-9151-3fa52e647581 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:49:45.909496 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d53bd36-2a18-40e1-9151-3fa52e647581 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:49:45.950811 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application [None req-5d53bd36-2a18-40e1-9151-3fa52e647581 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 11:49:45.950811 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:49:45.950811 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:49:45.950811 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:49:45.950811 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:49:45.950811 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:49:45.950811 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:49:45.950811 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:49:45.950811 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:49:45.950811 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:49:45.950811 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:49:45.950811 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:49:45.950811 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 11:49:45.950811 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 11:49:45.950811 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 11:49:45.950811 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 11:49:45.952790 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:49:45.952790 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:49:45.952790 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:49:45.952790 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:49:45.952790 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:49:45.952790 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:49:45.952790 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:49:45.952790 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application with Lock( Oct 11 11:49:45.952790 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:49:45.952790 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:49:45.952790 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:49:45.952790 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:49:45.952790 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:49:45.952790 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:49:45.952790 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:49:45.952790 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:49:45.952790 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 11:49:45.954724 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 11:49:45.954724 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 11:49:45.954724 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:49:45.959861 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 450/908] 10.222.0.59 () {56 vars in 1108 bytes} [Tue Oct 11 11:49:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:49:45.966924 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb21cf9-fbd0-47da-9301-478b2a975f35 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:49:46.173605 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb21cf9-fbd0-47da-9301-478b2a975f35 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:49:46.232157 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:49:46.244063 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc47203-6e6f-4529-b749-9ff76bec66ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:49:46.244765 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc47203-6e6f-4529-b749-9ff76bec66ed None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:49:46.245070 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc47203-6e6f-4529-b749-9ff76bec66ed None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:49:46.245349 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc47203-6e6f-4529-b749-9ff76bec66ed None admin] PATH_INFO: `/v3/domains` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:49:46.246227 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc47203-6e6f-4529-b749-9ff76bec66ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:49:46.246676 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc47203-6e6f-4529-b749-9ff76bec66ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:49:46.248495 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc47203-6e6f-4529-b749-9ff76bec66ed None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:49:46.266673 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 459/909] 10.222.0.59 () {56 vars in 1109 bytes} [Tue Oct 11 11:49:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:49:46.280302 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d53bd36-2a18-40e1-9151-3fa52e647581 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:49:46.365765 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:49:46.434669 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d53bd36-2a18-40e1-9151-3fa52e647581 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:49:46.480846 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5560c51d-bb66-4ef7-ba8b-e928447f8a60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:49:46.482009 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5560c51d-bb66-4ef7-ba8b-e928447f8a60 None admin] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:49:46.482758 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5560c51d-bb66-4ef7-ba8b-e928447f8a60 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:49:46.483584 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5560c51d-bb66-4ef7-ba8b-e928447f8a60 None admin] PATH_INFO: `/v3/users` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:49:46.484924 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5560c51d-bb66-4ef7-ba8b-e928447f8a60 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:49:46.487247 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5560c51d-bb66-4ef7-ba8b-e928447f8a60 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:49:46.533161 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 451/910] 10.222.0.59 () {60 vars in 1132 bytes} [Tue Oct 11 11:49:46 2022] POST /identity/v3/users => generated 251 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 11:49:49.238988 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1c12c8-57f6-4a6f-af85-25a4e2510010 None None] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:49:49.239775 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1c12c8-57f6-4a6f-af85-25a4e2510010 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:49:49.239775 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1c12c8-57f6-4a6f-af85-25a4e2510010 None None] PATH_INFO: `/` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:49:49.241433 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 460/911] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:49:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:49:49.250003 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187d230a-cc68-4641-ad6f-0014314c3ed5 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:49:49.250888 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187d230a-cc68-4641-ad6f-0014314c3ed5 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:49:49.251181 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187d230a-cc68-4641-ad6f-0014314c3ed5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:49:49.290103 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-187d230a-cc68-4641-ad6f-0014314c3ed5 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:49:49.292120 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-187d230a-cc68-4641-ad6f-0014314c3ed5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:49:49.393033 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 452/912] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:49:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 11:49:49.681138 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1c12c8-57f6-4a6f-af85-25a4e2510010 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:49:49.832667 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1c12c8-57f6-4a6f-af85-25a4e2510010 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:49:49.878476 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d272b3c-e56a-4e95-86b9-a6996d4514d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:49:49.879196 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d272b3c-e56a-4e95-86b9-a6996d4514d0 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:49:49.879492 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d272b3c-e56a-4e95-86b9-a6996d4514d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:49:49.879750 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d272b3c-e56a-4e95-86b9-a6996d4514d0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:49:49.898245 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d272b3c-e56a-4e95-86b9-a6996d4514d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 11:49:49.907412 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application [None req-8d272b3c-e56a-4e95-86b9-a6996d4514d0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 11 11:49:49.907412 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:49:49.907412 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:49:49.907412 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:49:49.907412 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:49:49.907412 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:49:49.907412 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:49:49.907412 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:49:49.907412 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:49:49.907412 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:49:49.907412 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:49:49.907412 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:49:49.907412 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 11:49:49.907412 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 11:49:49.907412 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 11:49:49.907412 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 11:49:49.909679 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 11:49:49.909679 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 11:49:49.909679 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 11:49:49.909679 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 11:49:49.909679 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:49:49.909679 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:49:49.909679 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:49:49.909679 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:49:49.909679 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:49:49.909679 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:49:49.909679 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:49:49.909679 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application with Lock( Oct 11 11:49:49.909679 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:49:49.909679 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:49:49.909679 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:49:49.909679 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:49:49.911302 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:49:49.911302 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:49:49.911302 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:49:49.911302 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:49:49.911302 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 11:49:49.911302 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 11:49:49.911302 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 11:49:49.911302 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 11:49:49.911302 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 11:49:49.911302 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 11:49:49.911302 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 11 11:49:49.911302 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:49:49.913780 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 461/913] 10.222.0.59 () {56 vars in 1102 bytes} [Tue Oct 11 11:49:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:49:49.919159 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187d230a-cc68-4641-ad6f-0014314c3ed5 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:49:50.002761 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:49:50.104473 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187d230a-cc68-4641-ad6f-0014314c3ed5 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:49:50.162628 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04eb8765-5d1b-4ddd-a906-5a1ce1372dec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:49:50.163315 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eb8765-5d1b-4ddd-a906-5a1ce1372dec None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:49:50.163643 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eb8765-5d1b-4ddd-a906-5a1ce1372dec None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:49:50.163868 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eb8765-5d1b-4ddd-a906-5a1ce1372dec None admin] PATH_INFO: `/v3/roles` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:49:50.164629 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04eb8765-5d1b-4ddd-a906-5a1ce1372dec None admin] RBAC: Adding query filter params (name=service) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:49:50.164936 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04eb8765-5d1b-4ddd-a906-5a1ce1372dec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:49:50.166372 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04eb8765-5d1b-4ddd-a906-5a1ce1372dec None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:49:50.179903 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 453/914] 10.222.0.59 () {56 vars in 1103 bytes} [Tue Oct 11 11:49:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:49:50.183594 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d272b3c-e56a-4e95-86b9-a6996d4514d0 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:49:50.265706 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:49:50.348452 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d272b3c-e56a-4e95-86b9-a6996d4514d0 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:49:50.408170 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3462d4-6d40-4a12-80fa-2c7279c37ade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:49:50.408928 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3462d4-6d40-4a12-80fa-2c7279c37ade None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:49:50.409232 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3462d4-6d40-4a12-80fa-2c7279c37ade None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:49:50.409523 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3462d4-6d40-4a12-80fa-2c7279c37ade None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:49:50.422818 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3462d4-6d40-4a12-80fa-2c7279c37ade None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:49:50.424341 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3462d4-6d40-4a12-80fa-2c7279c37ade None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:49:50.443747 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application [None req-bd3462d4-6d40-4a12-80fa-2c7279c37ade None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 11:49:50.443747 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:49:50.443747 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:49:50.443747 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:49:50.443747 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:49:50.443747 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:49:50.443747 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:49:50.443747 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:49:50.443747 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:49:50.443747 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:49:50.443747 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:49:50.443747 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:49:50.443747 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 11:49:50.443747 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 11:49:50.443747 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 11:49:50.443747 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 11:49:50.444805 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:49:50.444805 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:49:50.444805 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:49:50.444805 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:49:50.444805 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:49:50.444805 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:49:50.444805 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:49:50.444805 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application with Lock( Oct 11 11:49:50.444805 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:49:50.444805 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:49:50.444805 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:49:50.444805 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:49:50.444805 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:49:50.444805 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:49:50.444805 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:49:50.444805 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:49:50.444805 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 11:49:50.445843 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 11:49:50.445843 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 11:49:50.445843 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:49:50.445843 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 462/915] 10.222.0.59 () {56 vars in 1108 bytes} [Tue Oct 11 11:49:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:49:50.454345 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04eb8765-5d1b-4ddd-a906-5a1ce1372dec None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:49:50.650977 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:49:50.660010 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04eb8765-5d1b-4ddd-a906-5a1ce1372dec None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:49:50.716323 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a26506-0dde-4807-86f9-e446c00d085d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:49:50.716961 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a26506-0dde-4807-86f9-e446c00d085d None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:49:50.717225 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a26506-0dde-4807-86f9-e446c00d085d None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:49:50.717496 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a26506-0dde-4807-86f9-e446c00d085d None admin] PATH_INFO: `/v3/domains` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:49:50.718243 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a26506-0dde-4807-86f9-e446c00d085d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:49:50.718623 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a26506-0dde-4807-86f9-e446c00d085d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:49:50.720278 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a26506-0dde-4807-86f9-e446c00d085d None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:49:50.737436 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 454/916] 10.222.0.59 () {56 vars in 1109 bytes} [Tue Oct 11 11:49:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:49:50.744467 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3462d4-6d40-4a12-80fa-2c7279c37ade None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:49:50.913154 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3462d4-6d40-4a12-80fa-2c7279c37ade None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:49:50.916181 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:49:50.986786 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af994a20-a0a3-4de3-bb84-d8f975942f66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:49:50.988186 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af994a20-a0a3-4de3-bb84-d8f975942f66 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:49:50.988745 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af994a20-a0a3-4de3-bb84-d8f975942f66 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:49:50.989294 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af994a20-a0a3-4de3-bb84-d8f975942f66 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:49:51.005960 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af994a20-a0a3-4de3-bb84-d8f975942f66 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:49:51.009606 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af994a20-a0a3-4de3-bb84-d8f975942f66 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:49:51.030997 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application [None req-af994a20-a0a3-4de3-bb84-d8f975942f66 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 11 11:49:51.030997 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:49:51.030997 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:49:51.030997 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:49:51.030997 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:49:51.030997 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:49:51.030997 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:49:51.030997 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:49:51.030997 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:49:51.030997 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:49:51.030997 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:49:51.030997 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:49:51.030997 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 11 11:49:51.030997 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 11 11:49:51.030997 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 11 11:49:51.030997 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 11 11:49:51.032666 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:49:51.032666 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:49:51.032666 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 11 11:49:51.032666 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 11:49:51.032666 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 11 11:49:51.032666 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 11:49:51.032666 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:49:51.032666 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:49:51.032666 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:49:51.032666 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:49:51.032666 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:49:51.032666 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application with Lock( Oct 11 11:49:51.032666 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:49:51.032666 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:49:51.032666 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:49:51.032666 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:49:51.032666 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:49:51.032666 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:49:51.040361 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:49:51.040361 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:49:51.040361 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 11 11:49:51.040361 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 11 11:49:51.040361 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 11 11:49:51.040361 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 11 11:49:51.040361 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 11 11:49:51.040361 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 11 11:49:51.040361 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 11 11:49:51.040361 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:49:51.040361 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 463/917] 10.222.0.59 () {56 vars in 1108 bytes} [Tue Oct 11 11:49:50 2022] GET /identity/v3/users/placement => generated 87 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:49:51.041528 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a26506-0dde-4807-86f9-e446c00d085d None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:49:51.212087 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a26506-0dde-4807-86f9-e446c00d085d None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:49:51.276137 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079fa08c-c5c6-43ee-ac85-6648c4ad0033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:49:51.277489 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079fa08c-c5c6-43ee-ac85-6648c4ad0033 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:49:51.277961 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079fa08c-c5c6-43ee-ac85-6648c4ad0033 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:49:51.278448 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079fa08c-c5c6-43ee-ac85-6648c4ad0033 None admin] PATH_INFO: `/v3/users` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:49:51.279838 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079fa08c-c5c6-43ee-ac85-6648c4ad0033 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:49:51.280407 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079fa08c-c5c6-43ee-ac85-6648c4ad0033 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:49:51.283588 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079fa08c-c5c6-43ee-ac85-6648c4ad0033 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:49:51.319218 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 455/918] 10.222.0.59 () {56 vars in 1143 bytes} [Tue Oct 11 11:49:51 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 11:49:51.327914 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af994a20-a0a3-4de3-bb84-d8f975942f66 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:49:51.491633 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af994a20-a0a3-4de3-bb84-d8f975942f66 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:49:51.533873 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a481d53d-e3b8-4eb6-9589-c719b77b088a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:49:51.534918 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a481d53d-e3b8-4eb6-9589-c719b77b088a None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:49:51.535601 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a481d53d-e3b8-4eb6-9589-c719b77b088a None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:49:51.536029 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a481d53d-e3b8-4eb6-9589-c719b77b088a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:49:51.550654 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a481d53d-e3b8-4eb6-9589-c719b77b088a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:49:51.552477 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a481d53d-e3b8-4eb6-9589-c719b77b088a None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:49:51.577090 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application [None req-a481d53d-e3b8-4eb6-9589-c719b77b088a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 11:49:51.577090 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:49:51.577090 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:49:51.577090 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:49:51.577090 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:49:51.577090 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:49:51.577090 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:49:51.577090 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:49:51.577090 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:49:51.577090 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:49:51.577090 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:49:51.577090 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:49:51.577090 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 11:49:51.577090 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 11:49:51.577090 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 11:49:51.577090 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 11:49:51.578374 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:49:51.578374 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:49:51.578374 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:49:51.578374 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:49:51.578374 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:49:51.578374 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:49:51.578374 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:49:51.578374 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application with Lock( Oct 11 11:49:51.578374 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:49:51.578374 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:49:51.578374 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:49:51.578374 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:49:51.578374 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:49:51.578374 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:49:51.578374 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:49:51.578374 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:49:51.578374 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 11:49:51.584576 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 11:49:51.584576 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 11:49:51.584576 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:49:51.584576 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 464/919] 10.222.0.59 () {56 vars in 1108 bytes} [Tue Oct 11 11:49:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:49:51.585180 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079fa08c-c5c6-43ee-ac85-6648c4ad0033 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:49:51.609605 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:49:51.713042 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079fa08c-c5c6-43ee-ac85-6648c4ad0033 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:49:51.755944 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ff5f87-d132-4f74-bad0-bcd1fdc737e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:49:51.756596 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ff5f87-d132-4f74-bad0-bcd1fdc737e1 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:49:51.757009 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ff5f87-d132-4f74-bad0-bcd1fdc737e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:49:51.757425 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ff5f87-d132-4f74-bad0-bcd1fdc737e1 None admin] PATH_INFO: `/v3/domains` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:49:51.758261 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ff5f87-d132-4f74-bad0-bcd1fdc737e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:49:51.758704 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ff5f87-d132-4f74-bad0-bcd1fdc737e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:49:51.760403 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ff5f87-d132-4f74-bad0-bcd1fdc737e1 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:49:51.774503 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 456/920] 10.222.0.59 () {56 vars in 1109 bytes} [Tue Oct 11 11:49:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:49:51.785988 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a481d53d-e3b8-4eb6-9589-c719b77b088a None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:49:51.810027 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:49:51.941397 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a481d53d-e3b8-4eb6-9589-c719b77b088a None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:49:52.008813 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd03786-f8f2-440d-835c-873a340c71d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:49:52.009621 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd03786-f8f2-440d-835c-873a340c71d5 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:49:52.009933 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd03786-f8f2-440d-835c-873a340c71d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:49:52.010485 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd03786-f8f2-440d-835c-873a340c71d5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:49:52.019930 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd03786-f8f2-440d-835c-873a340c71d5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:49:52.021845 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd03786-f8f2-440d-835c-873a340c71d5 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:49:52.036833 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application [None req-8dd03786-f8f2-440d-835c-873a340c71d5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 11:49:52.036833 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:49:52.036833 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:49:52.036833 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:49:52.036833 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:49:52.036833 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:49:52.036833 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:49:52.036833 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:49:52.036833 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:49:52.036833 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:49:52.036833 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:49:52.036833 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:49:52.036833 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 11 11:49:52.036833 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 11 11:49:52.036833 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 11 11:49:52.038307 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 11 11:49:52.038307 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:49:52.038307 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:49:52.038307 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:49:52.038307 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:49:52.038307 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:49:52.038307 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:49:52.038307 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:49:52.038307 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application with Lock( Oct 11 11:49:52.038307 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:49:52.038307 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:49:52.038307 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:49:52.038307 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:49:52.038307 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:49:52.038307 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:49:52.038307 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:49:52.038307 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:49:52.039314 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 11 11:49:52.039314 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 11 11:49:52.039314 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 11 11:49:52.039314 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 11 11:49:52.039314 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 11 11:49:52.039314 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 11 11:49:52.039314 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 11:49:52.039314 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:49:52.039314 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 465/921] 10.222.0.59 () {56 vars in 1111 bytes} [Tue Oct 11 11:49:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:49:52.047426 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ff5f87-d132-4f74-bad0-bcd1fdc737e1 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:49:52.159330 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:49:52.194613 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ff5f87-d132-4f74-bad0-bcd1fdc737e1 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:49:52.241119 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1722b812-6a85-414c-82e3-18d16c57cfd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:49:52.241809 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1722b812-6a85-414c-82e3-18d16c57cfd8 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:49:52.242165 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1722b812-6a85-414c-82e3-18d16c57cfd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:49:52.242455 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1722b812-6a85-414c-82e3-18d16c57cfd8 None admin] PATH_INFO: `/v3/projects` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:49:52.243239 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1722b812-6a85-414c-82e3-18d16c57cfd8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:49:52.243514 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1722b812-6a85-414c-82e3-18d16c57cfd8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:49:52.244974 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1722b812-6a85-414c-82e3-18d16c57cfd8 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:49:52.263943 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 457/922] 10.222.0.59 () {56 vars in 1148 bytes} [Tue Oct 11 11:49:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:49:52.272289 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd03786-f8f2-440d-835c-873a340c71d5 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:49:52.398948 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:49:52.441040 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd03786-f8f2-440d-835c-873a340c71d5 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:49:52.491360 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb462af-3e86-4a6c-a019-d9fc54628513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:49:52.492143 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb462af-3e86-4a6c-a019-d9fc54628513 None admin] REQUEST_METHOD: `PUT` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:49:52.492470 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb462af-3e86-4a6c-a019-d9fc54628513 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:49:52.492804 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb462af-3e86-4a6c-a019-d9fc54628513 None admin] PATH_INFO: `/v3/projects/26ecf552cc9a4e5ca1d380509683dc4a/users/f272cdd1bcc04b4e8261ed533cfd4fd7/roles/f5a43d8f3fa24a87a7abe7492cf6f87f` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:49:52.514950 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb462af-3e86-4a6c-a019-d9fc54628513 None admin] RBAC: Authorizing `identity:create_grant(project_id=26ecf552cc9a4e5ca1d380509683dc4a, user_id=f272cdd1bcc04b4e8261ed533cfd4fd7, role_id=f5a43d8f3fa24a87a7abe7492cf6f87f)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:49:52.519402 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb462af-3e86-4a6c-a019-d9fc54628513 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:49:52.541747 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 466/923] 10.222.0.59 () {58 vars in 1439 bytes} [Tue Oct 11 11:49:52 2022] PUT /identity/v3/projects/26ecf552cc9a4e5ca1d380509683dc4a/users/f272cdd1bcc04b4e8261ed533cfd4fd7/roles/f5a43d8f3fa24a87a7abe7492cf6f87f => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 11:49:55.054399 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56df7914-4856-4649-ad8b-d39f4f64ae88 None None] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:49:55.055039 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56df7914-4856-4649-ad8b-d39f4f64ae88 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:49:55.055129 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56df7914-4856-4649-ad8b-d39f4f64ae88 None None] PATH_INFO: `/` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:49:55.056626 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 458/924] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:49:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 11 11:49:55.065917 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b966983-92bd-42e0-ad8d-7144918e7d0f None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:49:55.066525 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b966983-92bd-42e0-ad8d-7144918e7d0f None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:49:55.066865 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b966983-92bd-42e0-ad8d-7144918e7d0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:49:55.088777 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-7b966983-92bd-42e0-ad8d-7144918e7d0f None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:49:55.090019 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-7b966983-92bd-42e0-ad8d-7144918e7d0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:49:55.175049 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 467/925] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:49:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 11:49:55.447810 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56df7914-4856-4649-ad8b-d39f4f64ae88 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:49:55.570599 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56df7914-4856-4649-ad8b-d39f4f64ae88 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:49:55.612087 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67021891-ff57-4abe-b24d-1f289fe31982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:49:55.612751 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67021891-ff57-4abe-b24d-1f289fe31982 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:49:55.613091 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67021891-ff57-4abe-b24d-1f289fe31982 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:49:55.613535 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67021891-ff57-4abe-b24d-1f289fe31982 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:49:55.622480 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:49:55.629337 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67021891-ff57-4abe-b24d-1f289fe31982 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 11:49:55.635418 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application [None req-67021891-ff57-4abe-b24d-1f289fe31982 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 11 11:49:55.635418 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:49:55.635418 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:49:55.635418 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:49:55.635418 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:49:55.635418 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:49:55.635418 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:49:55.635418 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:49:55.635418 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:49:55.635418 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:49:55.635418 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:49:55.635418 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:49:55.635418 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 11:49:55.635418 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 11:49:55.635418 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 11:49:55.635418 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 11:49:55.636664 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 11:49:55.636664 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 11:49:55.636664 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 11:49:55.636664 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 11:49:55.636664 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:49:55.636664 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:49:55.636664 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:49:55.636664 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:49:55.636664 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:49:55.636664 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:49:55.636664 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:49:55.636664 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application with Lock( Oct 11 11:49:55.636664 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:49:55.636664 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:49:55.636664 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:49:55.636664 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:49:55.637653 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:49:55.637653 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:49:55.637653 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:49:55.637653 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:49:55.637653 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 11:49:55.637653 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 11:49:55.637653 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 11:49:55.637653 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 11:49:55.637653 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 11:49:55.637653 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 11:49:55.637653 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 11 11:49:55.637653 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:49:55.637653 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 459/926] 10.222.0.59 () {56 vars in 1102 bytes} [Tue Oct 11 11:49:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:49:55.644447 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b966983-92bd-42e0-ad8d-7144918e7d0f None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:49:55.734632 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:49:55.796807 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b966983-92bd-42e0-ad8d-7144918e7d0f None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:49:55.844089 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc59654-cb1d-487f-9647-a42e202862f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:49:55.845011 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc59654-cb1d-487f-9647-a42e202862f1 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:49:55.845478 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc59654-cb1d-487f-9647-a42e202862f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:49:55.845908 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc59654-cb1d-487f-9647-a42e202862f1 None admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:49:55.847104 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc59654-cb1d-487f-9647-a42e202862f1 None admin] RBAC: Adding query filter params (name=service) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:49:55.847631 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc59654-cb1d-487f-9647-a42e202862f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:49:55.849700 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc59654-cb1d-487f-9647-a42e202862f1 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:49:55.862592 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 468/927] 10.222.0.59 () {56 vars in 1103 bytes} [Tue Oct 11 11:49:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:49:55.869814 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67021891-ff57-4abe-b24d-1f289fe31982 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:49:55.985472 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67021891-ff57-4abe-b24d-1f289fe31982 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:49:56.025124 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8abef5-3d1c-4fe7-b878-38249e26c3bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:49:56.025797 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8abef5-3d1c-4fe7-b878-38249e26c3bb None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:49:56.026092 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8abef5-3d1c-4fe7-b878-38249e26c3bb None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:49:56.026349 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8abef5-3d1c-4fe7-b878-38249e26c3bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:49:56.038199 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8abef5-3d1c-4fe7-b878-38249e26c3bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:49:56.039722 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8abef5-3d1c-4fe7-b878-38249e26c3bb None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:49:56.056298 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application [None req-6e8abef5-3d1c-4fe7-b878-38249e26c3bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 11:49:56.056298 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:49:56.056298 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:49:56.056298 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:49:56.056298 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:49:56.056298 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:49:56.056298 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:49:56.056298 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:49:56.056298 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:49:56.056298 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:49:56.056298 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:49:56.056298 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:49:56.056298 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 11:49:56.056298 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 11:49:56.056298 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 11:49:56.056298 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 11:49:56.058042 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:49:56.058042 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:49:56.058042 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:49:56.058042 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:49:56.058042 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:49:56.058042 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:49:56.058042 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:49:56.058042 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application with Lock( Oct 11 11:49:56.058042 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:49:56.058042 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:49:56.058042 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:49:56.058042 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:49:56.058042 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:49:56.058042 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:49:56.058042 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:49:56.058042 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:49:56.058042 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 11:49:56.059304 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 11:49:56.059304 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 11:49:56.059304 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:49:56.059304 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 460/928] 10.222.0.59 () {56 vars in 1108 bytes} [Tue Oct 11 11:49:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 11 11:49:56.067239 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc59654-cb1d-487f-9647-a42e202862f1 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:49:56.204616 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc59654-cb1d-487f-9647-a42e202862f1 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:49:56.230100 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:49:56.252670 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a040120a-eb65-4857-9ef1-a511c97062b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:49:56.253359 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a040120a-eb65-4857-9ef1-a511c97062b2 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:49:56.253664 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a040120a-eb65-4857-9ef1-a511c97062b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:49:56.254007 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a040120a-eb65-4857-9ef1-a511c97062b2 None admin] PATH_INFO: `/v3/domains` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:49:56.254788 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a040120a-eb65-4857-9ef1-a511c97062b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:49:56.255167 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a040120a-eb65-4857-9ef1-a511c97062b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:49:56.256664 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a040120a-eb65-4857-9ef1-a511c97062b2 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:49:56.271854 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 469/929] 10.222.0.59 () {56 vars in 1109 bytes} [Tue Oct 11 11:49:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:49:56.278378 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8abef5-3d1c-4fe7-b878-38249e26c3bb None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:49:56.348246 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:49:56.411907 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8abef5-3d1c-4fe7-b878-38249e26c3bb None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:49:56.450465 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e4d141-0c6c-4423-a892-c0d1d9bedde9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:49:56.451250 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e4d141-0c6c-4423-a892-c0d1d9bedde9 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:49:56.451566 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e4d141-0c6c-4423-a892-c0d1d9bedde9 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:49:56.451850 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e4d141-0c6c-4423-a892-c0d1d9bedde9 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:49:56.463501 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e4d141-0c6c-4423-a892-c0d1d9bedde9 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:49:56.465191 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e4d141-0c6c-4423-a892-c0d1d9bedde9 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:49:56.480609 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application [None req-b6e4d141-0c6c-4423-a892-c0d1d9bedde9 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 11 11:49:56.480609 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:49:56.480609 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:49:56.480609 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:49:56.480609 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:49:56.480609 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:49:56.480609 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:49:56.480609 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:49:56.480609 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:49:56.480609 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:49:56.480609 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:49:56.480609 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:49:56.480609 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 11 11:49:56.480609 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 11 11:49:56.480609 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 11 11:49:56.480609 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 11 11:49:56.481876 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:49:56.481876 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:49:56.481876 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 11 11:49:56.481876 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 11:49:56.481876 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 11 11:49:56.481876 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 11:49:56.481876 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:49:56.481876 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:49:56.481876 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:49:56.481876 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:49:56.481876 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:49:56.481876 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application with Lock( Oct 11 11:49:56.481876 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:49:56.481876 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:49:56.481876 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:49:56.481876 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:49:56.481876 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:49:56.481876 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:49:56.483039 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:49:56.483039 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:49:56.483039 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 11 11:49:56.483039 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 11 11:49:56.483039 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 11 11:49:56.483039 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 11 11:49:56.483039 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 11 11:49:56.483039 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 11 11:49:56.483039 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 11 11:49:56.483039 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:49:56.483039 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 461/930] 10.222.0.59 () {56 vars in 1108 bytes} [Tue Oct 11 11:49:56 2022] GET /identity/v3/users/placement => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:49:56.499295 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a040120a-eb65-4857-9ef1-a511c97062b2 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:49:56.634160 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a040120a-eb65-4857-9ef1-a511c97062b2 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:49:56.682688 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956fe511-c292-4227-9aae-f23c9ba9dedd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:49:56.683518 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956fe511-c292-4227-9aae-f23c9ba9dedd None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:49:56.683905 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956fe511-c292-4227-9aae-f23c9ba9dedd None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:49:56.684186 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956fe511-c292-4227-9aae-f23c9ba9dedd None admin] PATH_INFO: `/v3/users` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:49:56.685276 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956fe511-c292-4227-9aae-f23c9ba9dedd None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:49:56.685862 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956fe511-c292-4227-9aae-f23c9ba9dedd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:49:56.688038 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956fe511-c292-4227-9aae-f23c9ba9dedd None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:49:56.712893 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 470/931] 10.222.0.59 () {56 vars in 1143 bytes} [Tue Oct 11 11:49:56 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:49:56.718994 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e4d141-0c6c-4423-a892-c0d1d9bedde9 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:49:56.839617 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e4d141-0c6c-4423-a892-c0d1d9bedde9 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:49:56.851978 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:49:56.887795 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec23203-2ead-4258-8220-92a938855409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:49:56.888458 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec23203-2ead-4258-8220-92a938855409 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:49:56.888794 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec23203-2ead-4258-8220-92a938855409 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:49:56.889065 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec23203-2ead-4258-8220-92a938855409 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:49:56.978013 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec23203-2ead-4258-8220-92a938855409 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:49:56.979540 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec23203-2ead-4258-8220-92a938855409 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:49:57.091733 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 462/932] 10.222.0.59 () {58 vars in 1282 bytes} [Tue Oct 11 11:49:56 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 11 11:49:57.099666 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956fe511-c292-4227-9aae-f23c9ba9dedd None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:49:57.155120 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:49:57.234724 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956fe511-c292-4227-9aae-f23c9ba9dedd None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:49:57.295268 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cd0429-81ee-4a4b-bd01-04bb3ecfeb8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:49:57.295934 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cd0429-81ee-4a4b-bd01-04bb3ecfeb8e None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:49:57.296210 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cd0429-81ee-4a4b-bd01-04bb3ecfeb8e None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:49:57.296483 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cd0429-81ee-4a4b-bd01-04bb3ecfeb8e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:49:57.308674 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cd0429-81ee-4a4b-bd01-04bb3ecfeb8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:49:57.310251 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cd0429-81ee-4a4b-bd01-04bb3ecfeb8e None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:49:57.330191 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application [None req-01cd0429-81ee-4a4b-bd01-04bb3ecfeb8e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 11:49:57.330191 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:49:57.330191 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:49:57.330191 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:49:57.330191 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:49:57.330191 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:49:57.330191 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:49:57.330191 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:49:57.330191 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:49:57.330191 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:49:57.330191 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:49:57.330191 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:49:57.330191 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 11:49:57.330191 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 11:49:57.330191 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 11:49:57.330191 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 11:49:57.332118 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:49:57.332118 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:49:57.332118 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:49:57.332118 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:49:57.332118 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:49:57.332118 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:49:57.332118 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:49:57.332118 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application with Lock( Oct 11 11:49:57.332118 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:49:57.332118 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:49:57.332118 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:49:57.332118 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:49:57.332118 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:49:57.332118 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:49:57.332118 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:49:57.332118 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:49:57.332118 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 11:49:57.334059 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 11:49:57.334059 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 11:49:57.334059 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:49:57.334059 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 471/933] 10.222.0.59 () {56 vars in 1108 bytes} [Tue Oct 11 11:49:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:49:57.338978 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec23203-2ead-4258-8220-92a938855409 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:49:57.355694 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:49:57.491410 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec23203-2ead-4258-8220-92a938855409 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:49:57.539245 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cabb0d-cc26-46ad-89b2-f675fc7733be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:49:57.540286 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cabb0d-cc26-46ad-89b2-f675fc7733be None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:49:57.540781 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cabb0d-cc26-46ad-89b2-f675fc7733be None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:49:57.541273 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cabb0d-cc26-46ad-89b2-f675fc7733be None admin] PATH_INFO: `/v3/domains` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:49:57.542449 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cabb0d-cc26-46ad-89b2-f675fc7733be None admin] RBAC: Adding query filter params (name=Default) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:49:57.543044 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cabb0d-cc26-46ad-89b2-f675fc7733be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:49:57.545617 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cabb0d-cc26-46ad-89b2-f675fc7733be None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:49:57.567640 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 463/934] 10.222.0.59 () {56 vars in 1109 bytes} [Tue Oct 11 11:49:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 11:49:57.576750 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cd0429-81ee-4a4b-bd01-04bb3ecfeb8e None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:49:57.703752 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:49:57.723686 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cd0429-81ee-4a4b-bd01-04bb3ecfeb8e None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:49:57.774664 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20403180-f7a3-4e6b-b376-c5f289cd29d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:49:57.775530 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20403180-f7a3-4e6b-b376-c5f289cd29d5 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:49:57.776027 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20403180-f7a3-4e6b-b376-c5f289cd29d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:49:57.776412 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20403180-f7a3-4e6b-b376-c5f289cd29d5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:49:57.785244 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20403180-f7a3-4e6b-b376-c5f289cd29d5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:49:57.787249 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20403180-f7a3-4e6b-b376-c5f289cd29d5 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:49:57.800725 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application [None req-20403180-f7a3-4e6b-b376-c5f289cd29d5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 11:49:57.800725 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:49:57.800725 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:49:57.800725 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:49:57.800725 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:49:57.800725 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:49:57.800725 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:49:57.800725 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:49:57.800725 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:49:57.800725 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:49:57.800725 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:49:57.800725 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:49:57.800725 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 11 11:49:57.800725 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 11 11:49:57.800725 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 11 11:49:57.801782 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 11 11:49:57.801782 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:49:57.801782 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:49:57.801782 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:49:57.801782 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:49:57.801782 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:49:57.801782 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:49:57.801782 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:49:57.801782 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application with Lock( Oct 11 11:49:57.801782 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:49:57.801782 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:49:57.801782 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:49:57.801782 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:49:57.801782 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:49:57.801782 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:49:57.801782 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:49:57.801782 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:49:57.802940 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 11 11:49:57.802940 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 11 11:49:57.802940 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 11 11:49:57.802940 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 11 11:49:57.802940 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 11 11:49:57.802940 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 11 11:49:57.802940 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 11:49:57.802940 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:49:57.802940 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 472/935] 10.222.0.59 () {56 vars in 1111 bytes} [Tue Oct 11 11:49:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:49:57.808162 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cabb0d-cc26-46ad-89b2-f675fc7733be None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:49:57.886739 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:49:57.935958 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cabb0d-cc26-46ad-89b2-f675fc7733be None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:49:57.983633 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d01476-ce70-479e-8102-d7b74569e910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:49:57.984286 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d01476-ce70-479e-8102-d7b74569e910 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:49:57.984666 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d01476-ce70-479e-8102-d7b74569e910 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:49:57.984992 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d01476-ce70-479e-8102-d7b74569e910 None admin] PATH_INFO: `/v3/projects` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:49:57.986010 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d01476-ce70-479e-8102-d7b74569e910 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:49:57.986324 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d01476-ce70-479e-8102-d7b74569e910 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:49:57.988376 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d01476-ce70-479e-8102-d7b74569e910 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:49:58.007440 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 464/936] 10.222.0.59 () {56 vars in 1148 bytes} [Tue Oct 11 11:49:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 11:49:58.013902 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20403180-f7a3-4e6b-b376-c5f289cd29d5 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:49:58.175062 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20403180-f7a3-4e6b-b376-c5f289cd29d5 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:49:58.218896 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66e7a23-a3c0-4617-a712-bf7fe4e75c0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:49:58.219727 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66e7a23-a3c0-4617-a712-bf7fe4e75c0d None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:49:58.220230 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66e7a23-a3c0-4617-a712-bf7fe4e75c0d None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:49:58.220547 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66e7a23-a3c0-4617-a712-bf7fe4e75c0d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:49:58.221646 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66e7a23-a3c0-4617-a712-bf7fe4e75c0d None admin] RBAC: Adding query filter params (role.id=f5a43d8f3fa24a87a7abe7492cf6f87f, scope.project.id=26ecf552cc9a4e5ca1d380509683dc4a, user.id=f272cdd1bcc04b4e8261ed533cfd4fd7) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:49:58.222025 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66e7a23-a3c0-4617-a712-bf7fe4e75c0d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:49:58.224780 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66e7a23-a3c0-4617-a712-bf7fe4e75c0d None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:49:58.231797 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 473/937] 10.222.0.59 () {56 vars in 1374 bytes} [Tue Oct 11 11:49:58 2022] GET /identity/v3/role_assignments?user.id=f272cdd1bcc04b4e8261ed533cfd4fd7&scope.project.id=26ecf552cc9a4e5ca1d380509683dc4a&role.id=f5a43d8f3fa24a87a7abe7492cf6f87f => generated 608 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:50:00.582533 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70c3258-2a4a-49cd-9768-09b26b0c4f20 None None] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:50:00.583145 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70c3258-2a4a-49cd-9768-09b26b0c4f20 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:50:00.583247 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70c3258-2a4a-49cd-9768-09b26b0c4f20 None None] PATH_INFO: `/` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:50:00.585468 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 465/938] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:50:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:50:00.591224 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7134abb-96bd-4400-80a1-829f54dc661c None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:50:00.591755 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7134abb-96bd-4400-80a1-829f54dc661c None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:50:00.591755 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7134abb-96bd-4400-80a1-829f54dc661c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:50:00.618034 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-b7134abb-96bd-4400-80a1-829f54dc661c None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:50:00.619270 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-b7134abb-96bd-4400-80a1-829f54dc661c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:50:00.622719 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:50:00.693871 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 474/939] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:50:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 11:50:00.985369 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70c3258-2a4a-49cd-9768-09b26b0c4f20 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:50:01.114774 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70c3258-2a4a-49cd-9768-09b26b0c4f20 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:50:01.151979 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:50:01.158653 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463701cf-2bdc-49f5-85ef-515e17d84df6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:50:01.159408 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463701cf-2bdc-49f5-85ef-515e17d84df6 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:50:01.159824 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463701cf-2bdc-49f5-85ef-515e17d84df6 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:50:01.160237 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463701cf-2bdc-49f5-85ef-515e17d84df6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:50:01.172626 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463701cf-2bdc-49f5-85ef-515e17d84df6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 11:50:01.177484 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application [None req-463701cf-2bdc-49f5-85ef-515e17d84df6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 11 11:50:01.177484 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:50:01.177484 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:50:01.177484 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:50:01.177484 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:50:01.177484 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:50:01.177484 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:50:01.177484 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:50:01.177484 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:50:01.177484 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:50:01.177484 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:50:01.177484 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:50:01.177484 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 11:50:01.177484 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 11:50:01.177484 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 11:50:01.177484 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 11:50:01.178714 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 11:50:01.178714 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 11:50:01.178714 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 11:50:01.178714 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 11:50:01.178714 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:50:01.178714 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:50:01.178714 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:50:01.178714 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:50:01.178714 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:50:01.178714 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:50:01.178714 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:50:01.178714 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application with Lock( Oct 11 11:50:01.178714 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:50:01.178714 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:50:01.178714 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:50:01.178714 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:50:01.179673 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:50:01.179673 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:50:01.179673 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:50:01.179673 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:50:01.179673 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 11:50:01.179673 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 11:50:01.179673 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 11:50:01.179673 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 11:50:01.179673 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 11:50:01.179673 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 11:50:01.179673 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 11 11:50:01.179673 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:50:01.179673 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 466/940] 10.222.0.59 () {56 vars in 1096 bytes} [Tue Oct 11 11:50:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:50:01.185273 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7134abb-96bd-4400-80a1-829f54dc661c None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:50:01.305456 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7134abb-96bd-4400-80a1-829f54dc661c None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:50:01.344947 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3110f581-5d17-47a5-ab60-3a47a4d631a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:50:01.345600 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3110f581-5d17-47a5-ab60-3a47a4d631a5 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:50:01.345892 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3110f581-5d17-47a5-ab60-3a47a4d631a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:50:01.346141 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3110f581-5d17-47a5-ab60-3a47a4d631a5 None admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:50:01.346855 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3110f581-5d17-47a5-ab60-3a47a4d631a5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:50:01.347187 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3110f581-5d17-47a5-ab60-3a47a4d631a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:50:01.348544 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3110f581-5d17-47a5-ab60-3a47a4d631a5 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:50:01.358119 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 475/941] 10.222.0.59 () {56 vars in 1099 bytes} [Tue Oct 11 11:50:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:50:01.367122 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463701cf-2bdc-49f5-85ef-515e17d84df6 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:50:01.521125 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463701cf-2bdc-49f5-85ef-515e17d84df6 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:50:01.572678 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7b9c4f-64c1-4920-b0b3-7318924fcb28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:50:01.573631 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7b9c4f-64c1-4920-b0b3-7318924fcb28 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:50:01.574103 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7b9c4f-64c1-4920-b0b3-7318924fcb28 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:50:01.574433 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7b9c4f-64c1-4920-b0b3-7318924fcb28 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:50:01.591164 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7b9c4f-64c1-4920-b0b3-7318924fcb28 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:50:01.592748 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7b9c4f-64c1-4920-b0b3-7318924fcb28 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:50:01.612528 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application [None req-af7b9c4f-64c1-4920-b0b3-7318924fcb28 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 11:50:01.612528 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:50:01.612528 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:50:01.612528 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:50:01.612528 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:50:01.612528 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:50:01.612528 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:50:01.612528 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:50:01.612528 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:50:01.612528 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:50:01.612528 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:50:01.612528 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:50:01.612528 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 11:50:01.612528 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 11:50:01.612528 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 11:50:01.612528 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 11:50:01.613721 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:50:01.613721 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:50:01.613721 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:50:01.613721 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:50:01.613721 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:50:01.613721 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:50:01.613721 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:50:01.613721 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application with Lock( Oct 11 11:50:01.613721 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:50:01.613721 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:50:01.613721 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:50:01.613721 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:50:01.613721 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:50:01.613721 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:50:01.613721 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:50:01.613721 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:50:01.613721 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 11:50:01.614770 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 11:50:01.614770 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 11:50:01.614770 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:50:01.614770 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 467/942] 10.222.0.59 () {56 vars in 1108 bytes} [Tue Oct 11 11:50:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:50:01.622172 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3110f581-5d17-47a5-ab60-3a47a4d631a5 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:50:01.626282 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:50:01.752665 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3110f581-5d17-47a5-ab60-3a47a4d631a5 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:50:01.797644 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84677caa-9496-4df9-9ac2-33a7d154f4f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:50:01.798660 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84677caa-9496-4df9-9ac2-33a7d154f4f8 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:50:01.798997 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84677caa-9496-4df9-9ac2-33a7d154f4f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:50:01.799533 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84677caa-9496-4df9-9ac2-33a7d154f4f8 None admin] PATH_INFO: `/v3/domains` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:50:01.800529 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84677caa-9496-4df9-9ac2-33a7d154f4f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:50:01.800856 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84677caa-9496-4df9-9ac2-33a7d154f4f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:50:01.802951 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84677caa-9496-4df9-9ac2-33a7d154f4f8 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:50:01.819122 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 476/943] 10.222.0.59 () {56 vars in 1109 bytes} [Tue Oct 11 11:50:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:50:01.826297 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7b9c4f-64c1-4920-b0b3-7318924fcb28 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:50:01.875415 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:50:01.952781 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7b9c4f-64c1-4920-b0b3-7318924fcb28 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:50:01.990537 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd8c7c3-fe37-47ec-84db-12428b529b0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:50:01.992374 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd8c7c3-fe37-47ec-84db-12428b529b0e None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:50:01.992374 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd8c7c3-fe37-47ec-84db-12428b529b0e None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:50:01.992374 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd8c7c3-fe37-47ec-84db-12428b529b0e None admin] PATH_INFO: `/v3/users/placement` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:50:02.001082 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd8c7c3-fe37-47ec-84db-12428b529b0e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:50:02.002564 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd8c7c3-fe37-47ec-84db-12428b529b0e None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:50:02.014896 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application [None req-1bd8c7c3-fe37-47ec-84db-12428b529b0e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 11 11:50:02.014896 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:50:02.014896 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:50:02.014896 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:50:02.014896 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:50:02.014896 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:50:02.014896 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:50:02.014896 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:50:02.014896 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:50:02.014896 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:50:02.014896 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:50:02.014896 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:50:02.014896 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 11 11:50:02.014896 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 11 11:50:02.014896 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 11 11:50:02.014896 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 11 11:50:02.016075 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:50:02.016075 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:50:02.016075 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 11 11:50:02.016075 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 11:50:02.016075 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 11 11:50:02.016075 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 11:50:02.016075 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:50:02.016075 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:50:02.016075 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:50:02.016075 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:50:02.016075 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:50:02.016075 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application with Lock( Oct 11 11:50:02.016075 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:50:02.016075 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:50:02.016075 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:50:02.016075 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:50:02.016075 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:50:02.016075 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:50:02.017173 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:50:02.017173 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:50:02.017173 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 11 11:50:02.017173 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 11 11:50:02.017173 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 11 11:50:02.017173 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 11 11:50:02.017173 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 11 11:50:02.017173 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 11 11:50:02.017173 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 11 11:50:02.017173 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:50:02.017173 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 468/944] 10.222.0.59 () {56 vars in 1108 bytes} [Tue Oct 11 11:50:01 2022] GET /identity/v3/users/placement => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:50:02.022583 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84677caa-9496-4df9-9ac2-33a7d154f4f8 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:50:02.107727 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:50:02.154182 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84677caa-9496-4df9-9ac2-33a7d154f4f8 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:50:02.194689 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c9f7fb-6655-4fb0-9695-fffd59a18848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:50:02.195411 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c9f7fb-6655-4fb0-9695-fffd59a18848 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:50:02.195887 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c9f7fb-6655-4fb0-9695-fffd59a18848 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:50:02.196238 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c9f7fb-6655-4fb0-9695-fffd59a18848 None admin] PATH_INFO: `/v3/users` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:50:02.197641 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c9f7fb-6655-4fb0-9695-fffd59a18848 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:50:02.198209 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c9f7fb-6655-4fb0-9695-fffd59a18848 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:50:02.200587 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c9f7fb-6655-4fb0-9695-fffd59a18848 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:50:02.224470 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 477/945] 10.222.0.59 () {56 vars in 1143 bytes} [Tue Oct 11 11:50:02 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:50:02.231925 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd8c7c3-fe37-47ec-84db-12428b529b0e None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:50:02.366142 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd8c7c3-fe37-47ec-84db-12428b529b0e None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:50:02.368502 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:50:02.416512 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b78801e-a545-4637-93df-df61004262ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:50:02.417340 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b78801e-a545-4637-93df-df61004262ff None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:50:02.417846 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b78801e-a545-4637-93df-df61004262ff None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:50:02.418185 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b78801e-a545-4637-93df-df61004262ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:50:02.435752 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b78801e-a545-4637-93df-df61004262ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:50:02.437633 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b78801e-a545-4637-93df-df61004262ff None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:50:02.455551 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application [None req-8b78801e-a545-4637-93df-df61004262ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 11:50:02.455551 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:50:02.455551 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:50:02.455551 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:50:02.455551 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:50:02.455551 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:50:02.455551 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:50:02.455551 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:50:02.455551 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:50:02.455551 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:50:02.455551 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:50:02.455551 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:50:02.455551 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 11:50:02.455551 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 11:50:02.455551 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 11:50:02.455551 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 11:50:02.457269 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:50:02.457269 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:50:02.457269 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:50:02.457269 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:50:02.457269 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:50:02.457269 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:50:02.457269 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:50:02.457269 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application with Lock( Oct 11 11:50:02.457269 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:50:02.457269 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:50:02.457269 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:50:02.457269 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:50:02.457269 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:50:02.457269 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:50:02.457269 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:50:02.457269 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:50:02.457269 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 11:50:02.458676 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 11:50:02.458676 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 11:50:02.458676 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:50:02.458676 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 469/946] 10.222.0.59 () {56 vars in 1108 bytes} [Tue Oct 11 11:50:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:50:02.462837 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c9f7fb-6655-4fb0-9695-fffd59a18848 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:50:02.601414 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c9f7fb-6655-4fb0-9695-fffd59a18848 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:50:02.627634 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:50:02.642816 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8dc02a-895a-4725-ba04-1b844b47db8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:50:02.643471 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8dc02a-895a-4725-ba04-1b844b47db8f None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:50:02.643727 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8dc02a-895a-4725-ba04-1b844b47db8f None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:50:02.644000 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8dc02a-895a-4725-ba04-1b844b47db8f None admin] PATH_INFO: `/v3/domains` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:50:02.644650 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8dc02a-895a-4725-ba04-1b844b47db8f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:50:02.644950 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8dc02a-895a-4725-ba04-1b844b47db8f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:50:02.646356 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8dc02a-895a-4725-ba04-1b844b47db8f None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:50:02.659886 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 478/947] 10.222.0.59 () {56 vars in 1109 bytes} [Tue Oct 11 11:50:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:50:02.668290 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b78801e-a545-4637-93df-df61004262ff None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:50:02.826632 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b78801e-a545-4637-93df-df61004262ff None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:50:02.869320 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dac33f7-6294-40b2-8e7c-5da0f0111122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:50:02.870088 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dac33f7-6294-40b2-8e7c-5da0f0111122 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:50:02.870571 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dac33f7-6294-40b2-8e7c-5da0f0111122 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:50:02.870911 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dac33f7-6294-40b2-8e7c-5da0f0111122 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:50:02.880365 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dac33f7-6294-40b2-8e7c-5da0f0111122 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:50:02.882419 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dac33f7-6294-40b2-8e7c-5da0f0111122 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:50:02.898907 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application [None req-9dac33f7-6294-40b2-8e7c-5da0f0111122 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 11:50:02.898907 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:50:02.898907 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:50:02.898907 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:50:02.898907 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:50:02.898907 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:50:02.898907 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:50:02.898907 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:50:02.898907 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:50:02.898907 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:50:02.898907 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:50:02.898907 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:50:02.898907 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 11 11:50:02.898907 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 11 11:50:02.898907 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 11 11:50:02.900097 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 11 11:50:02.900097 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:50:02.900097 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:50:02.900097 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:50:02.900097 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:50:02.900097 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:50:02.900097 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:50:02.900097 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:50:02.900097 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application with Lock( Oct 11 11:50:02.900097 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:50:02.900097 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:50:02.900097 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:50:02.900097 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:50:02.900097 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:50:02.900097 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:50:02.900097 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:50:02.900097 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:50:02.901230 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 11 11:50:02.901230 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 11 11:50:02.901230 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 11 11:50:02.901230 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 11 11:50:02.901230 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 11 11:50:02.901230 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 11 11:50:02.901230 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 11:50:02.901230 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:50:02.902037 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 470/948] 10.222.0.59 () {56 vars in 1111 bytes} [Tue Oct 11 11:50:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:50:02.907131 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8dc02a-895a-4725-ba04-1b844b47db8f None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:50:03.043600 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8dc02a-895a-4725-ba04-1b844b47db8f None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:50:03.087768 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e639aef7-4075-4368-bdd0-05b4595864e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:50:03.088664 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e639aef7-4075-4368-bdd0-05b4595864e8 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:50:03.089286 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e639aef7-4075-4368-bdd0-05b4595864e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:50:03.089835 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e639aef7-4075-4368-bdd0-05b4595864e8 None admin] PATH_INFO: `/v3/projects` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:50:03.090716 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e639aef7-4075-4368-bdd0-05b4595864e8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:50:03.091039 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e639aef7-4075-4368-bdd0-05b4595864e8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:50:03.092776 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e639aef7-4075-4368-bdd0-05b4595864e8 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:50:03.109188 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 479/949] 10.222.0.59 () {56 vars in 1148 bytes} [Tue Oct 11 11:50:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 11:50:03.116573 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dac33f7-6294-40b2-8e7c-5da0f0111122 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:50:03.139485 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:50:03.257156 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dac33f7-6294-40b2-8e7c-5da0f0111122 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:50:03.311404 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a175c03c-6bc3-4f45-ab43-55e5bc0f2dde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:50:03.312098 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a175c03c-6bc3-4f45-ab43-55e5bc0f2dde None admin] REQUEST_METHOD: `PUT` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:50:03.312385 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a175c03c-6bc3-4f45-ab43-55e5bc0f2dde None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:50:03.312656 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a175c03c-6bc3-4f45-ab43-55e5bc0f2dde None admin] PATH_INFO: `/v3/projects/26ecf552cc9a4e5ca1d380509683dc4a/users/f272cdd1bcc04b4e8261ed533cfd4fd7/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:50:03.314761 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a175c03c-6bc3-4f45-ab43-55e5bc0f2dde None admin] RBAC: Authorizing `identity:create_grant(project_id=26ecf552cc9a4e5ca1d380509683dc4a, user_id=f272cdd1bcc04b4e8261ed533cfd4fd7, role_id=5cf1f4bd2fef46ab9c6eff56d52f9f69)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:50:03.316377 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a175c03c-6bc3-4f45-ab43-55e5bc0f2dde None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:50:03.331884 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 471/950] 10.222.0.59 () {58 vars in 1439 bytes} [Tue Oct 11 11:50:03 2022] PUT /identity/v3/projects/26ecf552cc9a4e5ca1d380509683dc4a/users/f272cdd1bcc04b4e8261ed533cfd4fd7/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 11:50:05.551556 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3f173b-d353-4a7e-ba72-bc1e3edefa8d None None] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:50:05.552245 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3f173b-d353-4a7e-ba72-bc1e3edefa8d None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:50:05.552245 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3f173b-d353-4a7e-ba72-bc1e3edefa8d None None] PATH_INFO: `/` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:50:05.553723 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 480/951] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:50:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:50:05.561112 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab743ac-49f0-43f4-9ef9-d9e68f523076 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:50:05.561961 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab743ac-49f0-43f4-9ef9-d9e68f523076 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:50:05.561961 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab743ac-49f0-43f4-9ef9-d9e68f523076 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:50:05.586624 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-cab743ac-49f0-43f4-9ef9-d9e68f523076 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:50:05.587875 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-cab743ac-49f0-43f4-9ef9-d9e68f523076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:50:05.683937 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 472/952] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:50:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 11:50:05.973711 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3f173b-d353-4a7e-ba72-bc1e3edefa8d None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:50:06.040299 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:50:06.116044 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3f173b-d353-4a7e-ba72-bc1e3edefa8d None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:50:06.158824 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694c24d0-79ea-4abb-901d-721826ff4c53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:50:06.159456 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694c24d0-79ea-4abb-901d-721826ff4c53 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:50:06.159809 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694c24d0-79ea-4abb-901d-721826ff4c53 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:50:06.160116 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694c24d0-79ea-4abb-901d-721826ff4c53 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:50:06.173133 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694c24d0-79ea-4abb-901d-721826ff4c53 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 11 11:50:06.178209 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application [None req-694c24d0-79ea-4abb-901d-721826ff4c53 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 11 11:50:06.178209 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:50:06.178209 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:50:06.178209 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:50:06.178209 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:50:06.178209 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:50:06.178209 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:50:06.178209 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:50:06.178209 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:50:06.178209 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:50:06.178209 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:50:06.178209 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:50:06.178209 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 11 11:50:06.178209 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 11 11:50:06.178209 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 11 11:50:06.178209 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 11 11:50:06.180327 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 11 11:50:06.180327 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 11 11:50:06.180327 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 11 11:50:06.180327 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 11 11:50:06.180327 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:50:06.180327 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:50:06.180327 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:50:06.180327 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:50:06.180327 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:50:06.180327 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:50:06.180327 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:50:06.180327 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application with Lock( Oct 11 11:50:06.180327 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:50:06.180327 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:50:06.180327 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:50:06.180327 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:50:06.181654 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:50:06.181654 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:50:06.181654 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:50:06.181654 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:50:06.181654 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 11 11:50:06.181654 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 11 11:50:06.181654 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 11 11:50:06.181654 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 11 11:50:06.181654 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 11 11:50:06.181654 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 11 11:50:06.181654 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 11 11:50:06.181654 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:50:06.181654 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 481/953] 10.222.0.59 () {56 vars in 1096 bytes} [Tue Oct 11 11:50:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:50:06.186772 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab743ac-49f0-43f4-9ef9-d9e68f523076 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:50:06.190337 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:50:06.324694 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab743ac-49f0-43f4-9ef9-d9e68f523076 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:50:06.371648 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87975232-f253-45a7-9f59-556da41232a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:50:06.372809 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87975232-f253-45a7-9f59-556da41232a6 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:50:06.373314 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87975232-f253-45a7-9f59-556da41232a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:50:06.373763 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87975232-f253-45a7-9f59-556da41232a6 None admin] PATH_INFO: `/v3/roles` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:50:06.374607 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87975232-f253-45a7-9f59-556da41232a6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:50:06.374968 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87975232-f253-45a7-9f59-556da41232a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:50:06.376434 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87975232-f253-45a7-9f59-556da41232a6 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:50:06.386119 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 473/954] 10.222.0.59 () {56 vars in 1099 bytes} [Tue Oct 11 11:50:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:50:06.395236 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694c24d0-79ea-4abb-901d-721826ff4c53 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:50:06.529491 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:50:06.533614 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694c24d0-79ea-4abb-901d-721826ff4c53 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:50:06.592747 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a898d633-007f-46d9-883f-0d7687b72985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:50:06.593462 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a898d633-007f-46d9-883f-0d7687b72985 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:50:06.593721 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a898d633-007f-46d9-883f-0d7687b72985 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:50:06.593857 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a898d633-007f-46d9-883f-0d7687b72985 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:50:06.607771 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a898d633-007f-46d9-883f-0d7687b72985 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:50:06.609617 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a898d633-007f-46d9-883f-0d7687b72985 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:50:06.631104 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application [None req-a898d633-007f-46d9-883f-0d7687b72985 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 11:50:06.631104 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:50:06.631104 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:50:06.631104 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:50:06.631104 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:50:06.631104 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:50:06.631104 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:50:06.631104 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:50:06.631104 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:50:06.631104 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:50:06.631104 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:50:06.631104 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:50:06.631104 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 11:50:06.631104 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 11:50:06.631104 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 11:50:06.631104 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 11:50:06.632575 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:50:06.632575 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:50:06.632575 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:50:06.632575 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:50:06.632575 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:50:06.632575 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:50:06.632575 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:50:06.632575 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application with Lock( Oct 11 11:50:06.632575 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:50:06.632575 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:50:06.632575 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:50:06.632575 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:50:06.632575 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:50:06.632575 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:50:06.632575 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:50:06.632575 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:50:06.632575 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 11:50:06.634086 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 11:50:06.634086 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 11:50:06.634086 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:50:06.634086 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 482/955] 10.222.0.59 () {56 vars in 1108 bytes} [Tue Oct 11 11:50:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:50:06.638678 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87975232-f253-45a7-9f59-556da41232a6 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:50:06.714481 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:50:06.765342 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87975232-f253-45a7-9f59-556da41232a6 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:50:06.814218 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4515384-9e4a-4103-a720-817e767336e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:50:06.814909 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4515384-9e4a-4103-a720-817e767336e4 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:50:06.815186 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4515384-9e4a-4103-a720-817e767336e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:50:06.815456 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4515384-9e4a-4103-a720-817e767336e4 None admin] PATH_INFO: `/v3/domains` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:50:06.816109 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4515384-9e4a-4103-a720-817e767336e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:50:06.816438 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4515384-9e4a-4103-a720-817e767336e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:50:06.817904 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4515384-9e4a-4103-a720-817e767336e4 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:50:06.834600 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 474/956] 10.222.0.59 () {56 vars in 1109 bytes} [Tue Oct 11 11:50:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 11:50:06.839851 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a898d633-007f-46d9-883f-0d7687b72985 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:50:06.995370 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a898d633-007f-46d9-883f-0d7687b72985 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:50:07.043446 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad7a62f-dfef-48f8-b6a4-ab26b31e8e5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:50:07.044161 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad7a62f-dfef-48f8-b6a4-ab26b31e8e5e None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:50:07.044611 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad7a62f-dfef-48f8-b6a4-ab26b31e8e5e None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:50:07.044979 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad7a62f-dfef-48f8-b6a4-ab26b31e8e5e None admin] PATH_INFO: `/v3/users/placement` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:50:07.055558 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad7a62f-dfef-48f8-b6a4-ab26b31e8e5e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:50:07.057207 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad7a62f-dfef-48f8-b6a4-ab26b31e8e5e None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:50:07.071469 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application [None req-5ad7a62f-dfef-48f8-b6a4-ab26b31e8e5e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 11 11:50:07.071469 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:50:07.071469 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:50:07.071469 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:50:07.071469 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:50:07.071469 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:50:07.071469 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:50:07.071469 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:50:07.071469 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:50:07.071469 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:50:07.071469 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:50:07.071469 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:50:07.071469 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 11 11:50:07.071469 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 11 11:50:07.071469 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 11 11:50:07.071469 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 11 11:50:07.072665 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:50:07.072665 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:50:07.072665 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 11 11:50:07.072665 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 11:50:07.072665 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 11 11:50:07.072665 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 11 11:50:07.072665 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:50:07.072665 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:50:07.072665 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:50:07.072665 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:50:07.072665 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:50:07.072665 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application with Lock( Oct 11 11:50:07.072665 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:50:07.072665 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:50:07.072665 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:50:07.072665 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:50:07.072665 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:50:07.072665 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:50:07.073787 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:50:07.073787 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:50:07.073787 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 11 11:50:07.073787 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 11 11:50:07.073787 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 11 11:50:07.073787 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 11 11:50:07.073787 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 11 11:50:07.073787 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 11 11:50:07.073787 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 11 11:50:07.073787 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:50:07.073787 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 483/957] 10.222.0.59 () {56 vars in 1108 bytes} [Tue Oct 11 11:50:06 2022] GET /identity/v3/users/placement => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:50:07.078993 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4515384-9e4a-4103-a720-817e767336e4 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:50:07.200438 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4515384-9e4a-4103-a720-817e767336e4 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:50:07.228759 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:50:07.238077 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22618c40-d478-4764-9b48-fd68f7f4b0e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:50:07.239192 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22618c40-d478-4764-9b48-fd68f7f4b0e4 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:50:07.239192 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22618c40-d478-4764-9b48-fd68f7f4b0e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:50:07.239192 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22618c40-d478-4764-9b48-fd68f7f4b0e4 None admin] PATH_INFO: `/v3/users` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:50:07.239638 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22618c40-d478-4764-9b48-fd68f7f4b0e4 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:50:07.239938 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22618c40-d478-4764-9b48-fd68f7f4b0e4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:50:07.241380 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22618c40-d478-4764-9b48-fd68f7f4b0e4 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:50:07.260233 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 475/958] 10.222.0.59 () {56 vars in 1143 bytes} [Tue Oct 11 11:50:07 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:50:07.267514 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad7a62f-dfef-48f8-b6a4-ab26b31e8e5e None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:50:07.291954 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:50:07.411628 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad7a62f-dfef-48f8-b6a4-ab26b31e8e5e None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:50:07.457725 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bc86eb-885a-4bfe-ae8e-9e94be559d2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:50:07.458447 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bc86eb-885a-4bfe-ae8e-9e94be559d2e None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:50:07.458751 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bc86eb-885a-4bfe-ae8e-9e94be559d2e None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:50:07.459023 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bc86eb-885a-4bfe-ae8e-9e94be559d2e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:50:07.565407 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bc86eb-885a-4bfe-ae8e-9e94be559d2e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:50:07.568354 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bc86eb-885a-4bfe-ae8e-9e94be559d2e None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:50:07.593958 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:50:07.712218 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 484/959] 10.222.0.59 () {58 vars in 1282 bytes} [Tue Oct 11 11:50:07 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 11 11:50:07.718829 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22618c40-d478-4764-9b48-fd68f7f4b0e4 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:50:07.856038 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22618c40-d478-4764-9b48-fd68f7f4b0e4 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:50:07.908434 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be50503e-a750-4624-bfe0-244676e1d11a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:50:07.909130 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be50503e-a750-4624-bfe0-244676e1d11a None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:50:07.909405 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be50503e-a750-4624-bfe0-244676e1d11a None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:50:07.909682 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be50503e-a750-4624-bfe0-244676e1d11a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:50:07.930758 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be50503e-a750-4624-bfe0-244676e1d11a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:50:07.933431 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be50503e-a750-4624-bfe0-244676e1d11a None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:50:07.955496 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application [None req-be50503e-a750-4624-bfe0-244676e1d11a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 11:50:07.955496 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:50:07.955496 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:50:07.955496 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:50:07.955496 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:50:07.955496 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:50:07.955496 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:50:07.955496 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:50:07.955496 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:50:07.955496 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:50:07.955496 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:50:07.955496 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:50:07.955496 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 11 11:50:07.955496 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 11 11:50:07.955496 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 11 11:50:07.955496 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 11 11:50:07.957094 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:50:07.957094 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:50:07.957094 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:50:07.957094 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:50:07.957094 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:50:07.957094 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:50:07.957094 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:50:07.957094 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application with Lock( Oct 11 11:50:07.957094 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:50:07.957094 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:50:07.957094 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:50:07.957094 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:50:07.957094 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:50:07.957094 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:50:07.957094 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:50:07.957094 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:50:07.957094 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 11 11:50:07.958226 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 11 11:50:07.958226 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 11 11:50:07.958226 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:50:07.958226 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 476/960] 10.222.0.59 () {56 vars in 1108 bytes} [Tue Oct 11 11:50:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:50:07.962747 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bc86eb-885a-4bfe-ae8e-9e94be559d2e None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:50:08.099869 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bc86eb-885a-4bfe-ae8e-9e94be559d2e None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:50:08.116546 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:50:08.143562 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca2433d-9083-4959-8716-82ffa9126963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:50:08.144626 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca2433d-9083-4959-8716-82ffa9126963 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:50:08.144962 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca2433d-9083-4959-8716-82ffa9126963 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:50:08.145284 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca2433d-9083-4959-8716-82ffa9126963 None admin] PATH_INFO: `/v3/domains` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:50:08.146155 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca2433d-9083-4959-8716-82ffa9126963 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:50:08.146483 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca2433d-9083-4959-8716-82ffa9126963 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:50:08.148173 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca2433d-9083-4959-8716-82ffa9126963 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:50:08.164330 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 485/961] 10.222.0.59 () {56 vars in 1109 bytes} [Tue Oct 11 11:50:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:50:08.170363 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be50503e-a750-4624-bfe0-244676e1d11a None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:50:08.228399 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:50:08.311081 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be50503e-a750-4624-bfe0-244676e1d11a None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:50:08.353266 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687ba854-8a6b-48db-b0ac-abb5a588e3f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:50:08.354041 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687ba854-8a6b-48db-b0ac-abb5a588e3f4 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:50:08.354328 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687ba854-8a6b-48db-b0ac-abb5a588e3f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:50:08.354686 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687ba854-8a6b-48db-b0ac-abb5a588e3f4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:50:08.363449 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687ba854-8a6b-48db-b0ac-abb5a588e3f4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:50:08.365316 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687ba854-8a6b-48db-b0ac-abb5a588e3f4 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:50:08.381014 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application [None req-687ba854-8a6b-48db-b0ac-abb5a588e3f4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 11:50:08.381014 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:50:08.381014 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:50:08.381014 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:50:08.381014 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:50:08.381014 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:50:08.381014 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:50:08.381014 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:50:08.381014 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:50:08.381014 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:50:08.381014 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:50:08.381014 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:50:08.381014 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 11 11:50:08.381014 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 11 11:50:08.381014 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 11 11:50:08.382084 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 11 11:50:08.382084 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:50:08.382084 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:50:08.382084 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:50:08.382084 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:50:08.382084 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:50:08.382084 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:50:08.382084 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:50:08.382084 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application with Lock( Oct 11 11:50:08.382084 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:50:08.382084 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:50:08.382084 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:50:08.382084 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:50:08.382084 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:50:08.382084 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:50:08.382084 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:50:08.382084 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:50:08.383041 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 11 11:50:08.383041 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 11 11:50:08.383041 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 11 11:50:08.383041 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 11 11:50:08.383041 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 11 11:50:08.383041 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 11 11:50:08.383041 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 11 11:50:08.383041 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: ERROR keystone.server.flask.application Oct 11 11:50:08.383041 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 477/962] 10.222.0.59 () {56 vars in 1111 bytes} [Tue Oct 11 11:50:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:50:08.391808 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca2433d-9083-4959-8716-82ffa9126963 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:50:08.586752 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca2433d-9083-4959-8716-82ffa9126963 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:50:08.641804 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17216218-a17a-405a-a7d4-53e09ff048d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:50:08.642707 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17216218-a17a-405a-a7d4-53e09ff048d4 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:50:08.643006 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17216218-a17a-405a-a7d4-53e09ff048d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:50:08.643351 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17216218-a17a-405a-a7d4-53e09ff048d4 None admin] PATH_INFO: `/v3/projects` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:50:08.644643 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17216218-a17a-405a-a7d4-53e09ff048d4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:50:08.645122 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17216218-a17a-405a-a7d4-53e09ff048d4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:50:08.646912 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17216218-a17a-405a-a7d4-53e09ff048d4 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:50:08.665500 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 486/963] 10.222.0.59 () {56 vars in 1148 bytes} [Tue Oct 11 11:50:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 11:50:08.672393 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687ba854-8a6b-48db-b0ac-abb5a588e3f4 None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:50:08.793921 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:50:08.804351 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687ba854-8a6b-48db-b0ac-abb5a588e3f4 None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:50:08.867463 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602cfa00-778c-409e-b09d-d3409cd4b7c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:50:08.868138 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602cfa00-778c-409e-b09d-d3409cd4b7c2 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:50:08.868425 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602cfa00-778c-409e-b09d-d3409cd4b7c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:50:08.868688 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602cfa00-778c-409e-b09d-d3409cd4b7c2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:50:08.869490 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602cfa00-778c-409e-b09d-d3409cd4b7c2 None admin] RBAC: Adding query filter params (role.id=5cf1f4bd2fef46ab9c6eff56d52f9f69, scope.project.id=26ecf552cc9a4e5ca1d380509683dc4a, user.id=f272cdd1bcc04b4e8261ed533cfd4fd7) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:50:08.869818 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602cfa00-778c-409e-b09d-d3409cd4b7c2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:50:08.871535 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602cfa00-778c-409e-b09d-d3409cd4b7c2 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:50:08.878390 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 478/964] 10.222.0.59 () {56 vars in 1374 bytes} [Tue Oct 11 11:50:08 2022] GET /identity/v3/role_assignments?user.id=f272cdd1bcc04b4e8261ed533cfd4fd7&scope.project.id=26ecf552cc9a4e5ca1d380509683dc4a&role.id=5cf1f4bd2fef46ab9c6eff56d52f9f69 => generated 608 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:50:11.087176 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d14ef7-4c7e-41b9-b252-8b734426680f None None] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:50:11.087782 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d14ef7-4c7e-41b9-b252-8b734426680f None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:50:11.087782 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d14ef7-4c7e-41b9-b252-8b734426680f None None] PATH_INFO: `/` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:50:11.089223 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 487/965] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:50:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:50:11.095773 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05436e1c-7aab-42f7-878e-0514b788fb19 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:50:11.096330 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05436e1c-7aab-42f7-878e-0514b788fb19 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:50:11.096330 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05436e1c-7aab-42f7-878e-0514b788fb19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:50:11.116410 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-05436e1c-7aab-42f7-878e-0514b788fb19 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:50:11.117649 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-05436e1c-7aab-42f7-878e-0514b788fb19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:50:11.191138 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 479/966] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:50:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 11:50:11.452874 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d14ef7-4c7e-41b9-b252-8b734426680f None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:50:11.487216 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:50:11.569241 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d14ef7-4c7e-41b9-b252-8b734426680f None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:50:11.606842 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3754de-33c0-4fa3-b308-6bb10af4381c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:50:11.607480 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3754de-33c0-4fa3-b308-6bb10af4381c None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:50:11.607756 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3754de-33c0-4fa3-b308-6bb10af4381c None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:50:11.608025 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3754de-33c0-4fa3-b308-6bb10af4381c None admin] PATH_INFO: `/v3/services/placement` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:50:11.608605 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3754de-33c0-4fa3-b308-6bb10af4381c None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:50:11.609982 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3754de-33c0-4fa3-b308-6bb10af4381c None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:50:11.618990 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application [None req-1d3754de-33c0-4fa3-b308-6bb10af4381c None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 11 11:50:11.618990 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:50:11.618990 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 11 11:50:11.618990 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 11 11:50:11.618990 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 11 11:50:11.618990 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 11 11:50:11.618990 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 11 11:50:11.618990 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 11:50:11.618990 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 11 11:50:11.618990 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:50:11.618990 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 11 11:50:11.618990 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:50:11.618990 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:50:11.618990 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:50:11.618990 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:50:11.618990 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:50:11.618990 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:50:11.622715 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:50:11.622715 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:50:11.622715 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:50:11.622715 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:50:11.622715 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:50:11.622715 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:50:11.622715 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 11 11:50:11.622715 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 11 11:50:11.622715 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 11 11:50:11.622715 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 11 11:50:11.622715 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:50:11.622715 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:50:11.622715 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:50:11.622715 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:50:11.622715 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:50:11.622715 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:50:11.623800 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:50:11.623800 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application with Lock( Oct 11 11:50:11.623800 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:50:11.623800 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:50:11.623800 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:50:11.623800 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:50:11.623800 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:50:11.623800 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:50:11.623800 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:50:11.623800 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:50:11.623800 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 11 11:50:11.623800 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 11:50:11.623800 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 11 11:50:11.623800 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:50:11.623800 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 488/967] 10.222.0.59 () {56 vars in 1117 bytes} [Tue Oct 11 11:50:11 2022] GET /identity/v3/services/placement => generated 90 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:50:11.626372 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05436e1c-7aab-42f7-878e-0514b788fb19 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:50:11.720427 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:50:11.731179 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05436e1c-7aab-42f7-878e-0514b788fb19 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:50:11.767202 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93db644-dc77-4538-a4ad-59418ba7a619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:50:11.767811 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93db644-dc77-4538-a4ad-59418ba7a619 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:50:11.768088 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93db644-dc77-4538-a4ad-59418ba7a619 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:50:11.768353 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93db644-dc77-4538-a4ad-59418ba7a619 None admin] PATH_INFO: `/v3/services` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:50:11.768965 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93db644-dc77-4538-a4ad-59418ba7a619 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:50:11.769262 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93db644-dc77-4538-a4ad-59418ba7a619 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:50:11.770661 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93db644-dc77-4538-a4ad-59418ba7a619 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:50:11.775476 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 480/968] 10.222.0.59 () {56 vars in 1116 bytes} [Tue Oct 11 11:50:11 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:50:11.781873 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3754de-33c0-4fa3-b308-6bb10af4381c None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:50:11.876215 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:50:11.882590 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3754de-33c0-4fa3-b308-6bb10af4381c None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:50:11.921282 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e0b5b0-7956-4a38-9eec-2a5156c57a55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:50:11.921898 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e0b5b0-7956-4a38-9eec-2a5156c57a55 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:50:11.922178 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e0b5b0-7956-4a38-9eec-2a5156c57a55 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:50:11.922447 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e0b5b0-7956-4a38-9eec-2a5156c57a55 None admin] PATH_INFO: `/v3/services` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:50:11.923111 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e0b5b0-7956-4a38-9eec-2a5156c57a55 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:50:11.923407 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e0b5b0-7956-4a38-9eec-2a5156c57a55 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:50:11.924806 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e0b5b0-7956-4a38-9eec-2a5156c57a55 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:50:11.929992 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 489/969] 10.222.0.59 () {56 vars in 1116 bytes} [Tue Oct 11 11:50:11 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:50:14.052941 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47eb5758-b9b5-464f-88bf-e1fd0ec8e346 None None] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:50:14.053504 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47eb5758-b9b5-464f-88bf-e1fd0ec8e346 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:50:14.053504 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47eb5758-b9b5-464f-88bf-e1fd0ec8e346 None None] PATH_INFO: `/` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:50:14.054870 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 481/970] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:50:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:50:14.060928 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01e834b-c264-4069-8d03-845f81ea0dbc None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:50:14.061452 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01e834b-c264-4069-8d03-845f81ea0dbc None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:50:14.061452 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01e834b-c264-4069-8d03-845f81ea0dbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:50:14.081983 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-f01e834b-c264-4069-8d03-845f81ea0dbc None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:50:14.083257 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-f01e834b-c264-4069-8d03-845f81ea0dbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:50:14.156889 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 490/971] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:50:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 11:50:14.439624 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47eb5758-b9b5-464f-88bf-e1fd0ec8e346 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:50:14.568959 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47eb5758-b9b5-464f-88bf-e1fd0ec8e346 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:50:14.610332 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873a9e1f-2af4-4fbc-9232-7e54ebba4472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:50:14.610988 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873a9e1f-2af4-4fbc-9232-7e54ebba4472 None admin] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:50:14.611259 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873a9e1f-2af4-4fbc-9232-7e54ebba4472 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:50:14.611532 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873a9e1f-2af4-4fbc-9232-7e54ebba4472 None admin] PATH_INFO: `/v3/services` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:50:14.612193 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873a9e1f-2af4-4fbc-9232-7e54ebba4472 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:50:14.613557 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873a9e1f-2af4-4fbc-9232-7e54ebba4472 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:50:14.630594 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 482/972] 10.222.0.59 () {60 vars in 1141 bytes} [Tue Oct 11 11:50:14 2022] POST /identity/v3/services => generated 246 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 11:50:16.757752 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a95cbf-b26a-4136-b11d-e4c60265163a None None] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:50:16.758328 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a95cbf-b26a-4136-b11d-e4c60265163a None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:50:16.758328 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a95cbf-b26a-4136-b11d-e4c60265163a None None] PATH_INFO: `/` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:50:16.759686 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 491/973] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:50:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:50:16.765813 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932b63ca-3da5-44a7-a95a-87990e375bcc None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:50:16.766366 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932b63ca-3da5-44a7-a95a-87990e375bcc None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:50:16.766366 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932b63ca-3da5-44a7-a95a-87990e375bcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:50:16.787009 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-932b63ca-3da5-44a7-a95a-87990e375bcc None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:50:16.788198 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-932b63ca-3da5-44a7-a95a-87990e375bcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:50:16.799346 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:50:16.879198 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 483/974] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:50:16 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 11:50:17.141442 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a95cbf-b26a-4136-b11d-e4c60265163a None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:50:17.238802 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:50:17.250598 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a95cbf-b26a-4136-b11d-e4c60265163a None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:50:17.286151 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2b2022-9c8b-4a98-be67-7b104c5ec359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:50:17.286811 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2b2022-9c8b-4a98-be67-7b104c5ec359 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:50:17.287088 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2b2022-9c8b-4a98-be67-7b104c5ec359 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:50:17.287357 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2b2022-9c8b-4a98-be67-7b104c5ec359 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:50:17.287933 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2b2022-9c8b-4a98-be67-7b104c5ec359 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:50:17.289303 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2b2022-9c8b-4a98-be67-7b104c5ec359 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:50:17.298359 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application [None req-2c2b2022-9c8b-4a98-be67-7b104c5ec359 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 11 11:50:17.298359 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:50:17.298359 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 11 11:50:17.298359 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 11 11:50:17.298359 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 11 11:50:17.298359 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 11 11:50:17.298359 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 11 11:50:17.298359 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 11:50:17.298359 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 11 11:50:17.298359 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:50:17.298359 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 11 11:50:17.298359 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:50:17.298359 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:50:17.298359 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:50:17.298359 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:50:17.298359 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:50:17.298359 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:50:17.299436 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:50:17.299436 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:50:17.299436 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:50:17.299436 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:50:17.299436 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:50:17.299436 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:50:17.299436 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 11 11:50:17.299436 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 11 11:50:17.299436 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 11 11:50:17.299436 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 11 11:50:17.299436 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:50:17.299436 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:50:17.299436 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:50:17.299436 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:50:17.299436 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:50:17.299436 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:50:17.300273 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:50:17.300273 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application with Lock( Oct 11 11:50:17.300273 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:50:17.300273 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:50:17.300273 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:50:17.300273 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:50:17.300273 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:50:17.300273 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:50:17.300273 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:50:17.300273 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:50:17.300273 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 11 11:50:17.300273 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 11:50:17.300273 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 11 11:50:17.300273 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:50:17.300273 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 492/975] 10.222.0.59 () {56 vars in 1117 bytes} [Tue Oct 11 11:50:17 2022] GET /identity/v3/services/placement => generated 90 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:50:17.306067 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932b63ca-3da5-44a7-a95a-87990e375bcc None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:50:17.411015 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932b63ca-3da5-44a7-a95a-87990e375bcc None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:50:17.446538 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e7872b-fa05-4dc1-8b6a-6e2e1786a7cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:50:17.447250 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e7872b-fa05-4dc1-8b6a-6e2e1786a7cc None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:50:17.447535 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e7872b-fa05-4dc1-8b6a-6e2e1786a7cc None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:50:17.447809 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e7872b-fa05-4dc1-8b6a-6e2e1786a7cc None admin] PATH_INFO: `/v3/services` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:50:17.448504 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e7872b-fa05-4dc1-8b6a-6e2e1786a7cc None admin] RBAC: Adding query filter params (name=placement) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:50:17.448814 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e7872b-fa05-4dc1-8b6a-6e2e1786a7cc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:50:17.450186 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e7872b-fa05-4dc1-8b6a-6e2e1786a7cc None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:50:17.455522 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 484/976] 10.222.0.59 () {56 vars in 1116 bytes} [Tue Oct 11 11:50:17 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:50:17.461989 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2b2022-9c8b-4a98-be67-7b104c5ec359 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:50:17.578949 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2b2022-9c8b-4a98-be67-7b104c5ec359 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:50:17.616757 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a83174-0d6b-4a06-b1da-193057bb2a6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:50:17.617400 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a83174-0d6b-4a06-b1da-193057bb2a6f None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:50:17.617687 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a83174-0d6b-4a06-b1da-193057bb2a6f None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:50:17.617965 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a83174-0d6b-4a06-b1da-193057bb2a6f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:50:17.618758 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a83174-0d6b-4a06-b1da-193057bb2a6f None admin] RBAC: Adding query filter params (interface=public, service_id=956b68b9b89944b08ddd298d8812d409, region_id=RegionOne) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:50:17.619080 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a83174-0d6b-4a06-b1da-193057bb2a6f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:50:17.620631 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a83174-0d6b-4a06-b1da-193057bb2a6f None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:50:17.626123 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 493/977] 10.222.0.59 () {56 vars in 1251 bytes} [Tue Oct 11 11:50:17 2022] GET /identity/v3/endpoints?service_id=956b68b9b89944b08ddd298d8812d409&interface=public®ion_id=RegionOne => generated 194 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:50:17.632907 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e7872b-fa05-4dc1-8b6a-6e2e1786a7cc None admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:50:17.637596 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:50:17.738106 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e7872b-fa05-4dc1-8b6a-6e2e1786a7cc None admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:50:17.774538 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aca4189-1f50-419c-b1d8-92b82a426f0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:50:17.775337 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aca4189-1f50-419c-b1d8-92b82a426f0e None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:50:17.775534 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aca4189-1f50-419c-b1d8-92b82a426f0e None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:50:17.775811 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aca4189-1f50-419c-b1d8-92b82a426f0e None admin] PATH_INFO: `/v3/services` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:50:17.776545 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aca4189-1f50-419c-b1d8-92b82a426f0e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:50:17.778143 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aca4189-1f50-419c-b1d8-92b82a426f0e None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:50:17.785518 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 485/978] 10.222.0.59 () {56 vars in 1087 bytes} [Tue Oct 11 11:50:17 2022] GET /identity/v3/services => generated 2191 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 11:50:19.926113 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6039bf4-de04-4547-9dbc-b0afd9c62ebd None None] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:50:19.926724 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6039bf4-de04-4547-9dbc-b0afd9c62ebd None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:50:19.926724 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6039bf4-de04-4547-9dbc-b0afd9c62ebd None None] PATH_INFO: `/` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:50:19.928045 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 494/979] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:50:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:50:19.933971 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b6efe3-c3e6-413f-b496-941da880b474 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:50:19.934549 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b6efe3-c3e6-413f-b496-941da880b474 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:50:19.934549 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b6efe3-c3e6-413f-b496-941da880b474 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:50:19.955226 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-76b6efe3-c3e6-413f-b496-941da880b474 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:50:19.956468 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-76b6efe3-c3e6-413f-b496-941da880b474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:50:20.027245 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:50:20.034961 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 486/980] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:50:19 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 11 11:50:20.306247 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6039bf4-de04-4547-9dbc-b0afd9c62ebd None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:50:20.327648 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:50:20.414381 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6039bf4-de04-4547-9dbc-b0afd9c62ebd None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:50:20.449028 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d57e915-2185-4a12-9b51-9dc68ad224f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:50:20.449635 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d57e915-2185-4a12-9b51-9dc68ad224f9 None admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:50:20.449905 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d57e915-2185-4a12-9b51-9dc68ad224f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:50:20.450182 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d57e915-2185-4a12-9b51-9dc68ad224f9 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:50:20.450786 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d57e915-2185-4a12-9b51-9dc68ad224f9 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:50:20.452188 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d57e915-2185-4a12-9b51-9dc68ad224f9 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:50:20.461196 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application [None req-2d57e915-2185-4a12-9b51-9dc68ad224f9 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 11 11:50:20.461196 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:50:20.461196 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 11 11:50:20.461196 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 11 11:50:20.461196 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 11 11:50:20.461196 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 11 11:50:20.461196 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 11 11:50:20.461196 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 11:50:20.461196 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 11 11:50:20.461196 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:50:20.461196 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 11 11:50:20.461196 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:50:20.461196 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 11 11:50:20.461196 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 11 11:50:20.461196 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 11 11:50:20.461196 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 11 11:50:20.461196 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 11 11:50:20.462455 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 11 11:50:20.462455 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 11 11:50:20.462455 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 11 11:50:20.462455 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 11 11:50:20.462455 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 11 11:50:20.462455 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 11 11:50:20.462455 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 11 11:50:20.462455 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 11 11:50:20.462455 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 11 11:50:20.462455 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 11 11:50:20.462455 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 11 11:50:20.462455 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 11 11:50:20.462455 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 11 11:50:20.462455 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 11 11:50:20.462455 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 11 11:50:20.462455 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.get_or_create( Oct 11 11:50:20.463406 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 11 11:50:20.463406 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application with Lock( Oct 11 11:50:20.463406 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 11 11:50:20.463406 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self._enter() Oct 11 11:50:20.463406 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 11 11:50:20.463406 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 11 11:50:20.463406 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 11 11:50:20.463406 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application return self.creator() Oct 11 11:50:20.463406 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 11 11:50:20.463406 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application created_value = creator( Oct 11 11:50:20.463406 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 11 11:50:20.463406 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 11 11:50:20.463406 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 11 11:50:20.463406 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: ERROR keystone.server.flask.application Oct 11 11:50:20.463406 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 495/981] 10.222.0.59 () {56 vars in 1117 bytes} [Tue Oct 11 11:50:20 2022] GET /identity/v3/services/placement => generated 90 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 11 11:50:20.468540 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b6efe3-c3e6-413f-b496-941da880b474 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:50:20.582822 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b6efe3-c3e6-413f-b496-941da880b474 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:50:20.617465 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952eb158-a1d8-4b0c-a937-15dbe965aef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:50:20.618069 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952eb158-a1d8-4b0c-a937-15dbe965aef7 None admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:50:20.618340 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952eb158-a1d8-4b0c-a937-15dbe965aef7 None admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:50:20.618653 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952eb158-a1d8-4b0c-a937-15dbe965aef7 None admin] PATH_INFO: `/v3/services` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:50:20.619294 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952eb158-a1d8-4b0c-a937-15dbe965aef7 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:50:20.619614 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952eb158-a1d8-4b0c-a937-15dbe965aef7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:50:20.620976 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952eb158-a1d8-4b0c-a937-15dbe965aef7 None admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:50:20.626261 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 487/982] 10.222.0.59 () {56 vars in 1116 bytes} [Tue Oct 11 11:50:20 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:50:20.632474 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d57e915-2185-4a12-9b51-9dc68ad224f9 None admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:50:20.720454 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:50:20.742897 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d57e915-2185-4a12-9b51-9dc68ad224f9 None admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:50:20.778979 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fd26e8-ca29-471d-9026-7ad016f6e6e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:50:20.779579 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fd26e8-ca29-471d-9026-7ad016f6e6e4 None admin] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:50:20.779854 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fd26e8-ca29-471d-9026-7ad016f6e6e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:50:20.780107 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fd26e8-ca29-471d-9026-7ad016f6e6e4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:50:20.781074 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fd26e8-ca29-471d-9026-7ad016f6e6e4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:50:20.782447 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fd26e8-ca29-471d-9026-7ad016f6e6e4 None admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:50:20.802970 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 496/983] 10.222.0.59 () {60 vars in 1144 bytes} [Tue Oct 11 11:50:20 2022] POST /identity/v3/endpoints => generated 331 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 11:50:30.623112 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=69 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:50:30.798488 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=30,keystone:INSERT=1 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:51:30.035802 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb6d201-c092-495b-8855-b8962b34f045 None None] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:51:30.036570 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb6d201-c092-495b-8855-b8962b34f045 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:51:30.036570 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb6d201-c092-495b-8855-b8962b34f045 None None] PATH_INFO: `/` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:51:30.037757 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 488/984] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:51:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:51:30.044775 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebca5f0-2ec6-48c0-9355-2127f344cf4e None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:51:30.045456 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebca5f0-2ec6-48c0-9355-2127f344cf4e None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:51:30.045875 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebca5f0-2ec6-48c0-9355-2127f344cf4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:51:30.181310 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-cebca5f0-2ec6-48c0-9355-2127f344cf4e None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:51:30.183346 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-cebca5f0-2ec6-48c0-9355-2127f344cf4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:51:30.363514 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 497/985] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:51:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 11:51:30.885879 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd37aa7-2643-4044-a975-84ef6241f98d None None] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:51:30.886143 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd37aa7-2643-4044-a975-84ef6241f98d None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:51:30.886539 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd37aa7-2643-4044-a975-84ef6241f98d None None] PATH_INFO: `/` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:51:30.888149 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 489/986] 10.222.0.59 () {54 vars in 964 bytes} [Tue Oct 11 11:51:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 11 11:51:30.894867 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef94fa0d-d37c-4348-92ba-223775397145 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:51:30.895140 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef94fa0d-d37c-4348-92ba-223775397145 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:51:30.895561 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef94fa0d-d37c-4348-92ba-223775397145 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:51:31.041251 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-ef94fa0d-d37c-4348-92ba-223775397145 None None] MFA Rules not processed for user `2e563fe174db4cc19ae6bf8162cc3d23`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:51:31.042093 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-ef94fa0d-d37c-4348-92ba-223775397145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:51:31.070983 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:51:31.187529 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 498/987] 10.222.0.59 () {58 vars in 1061 bytes} [Tue Oct 11 11:51:30 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 11:51:31.197189 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd37aa7-2643-4044-a975-84ef6241f98d None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:51:31.200366 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-9dd37aa7-2643-4044-a975-84ef6241f98d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:51:31.404168 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd37aa7-2643-4044-a975-84ef6241f98d None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:51:31.529168 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736f3c87-49b3-4d1d-b5e9-0e1ea11c5017 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e563fe174db4cc19ae6bf8162cc3d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:51:31.529811 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736f3c87-49b3-4d1d-b5e9-0e1ea11c5017 service neutron] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:51:31.530095 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736f3c87-49b3-4d1d-b5e9-0e1ea11c5017 service neutron] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:51:31.530368 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736f3c87-49b3-4d1d-b5e9-0e1ea11c5017 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:51:31.532261 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-736f3c87-49b3-4d1d-b5e9-0e1ea11c5017 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:51:31.614169 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:51:31.681639 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736f3c87-49b3-4d1d-b5e9-0e1ea11c5017 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:51:31.683047 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736f3c87-49b3-4d1d-b5e9-0e1ea11c5017 service neutron] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:51:31.705874 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 490/988] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 11:51:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 510 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 11 11:51:34.271777 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dae0df7-5e90-4f33-b1b6-71e787332e80 None None] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:51:34.272486 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dae0df7-5e90-4f33-b1b6-71e787332e80 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:51:34.272486 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dae0df7-5e90-4f33-b1b6-71e787332e80 None None] PATH_INFO: `/` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:51:34.273977 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 499/989] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:51:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:51:34.280815 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a98266-5171-4070-8b57-26e3b32f94ef None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:51:34.281498 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a98266-5171-4070-8b57-26e3b32f94ef None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:51:34.281498 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a98266-5171-4070-8b57-26e3b32f94ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:51:34.400546 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-32a98266-5171-4070-8b57-26e3b32f94ef None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:51:34.401850 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-32a98266-5171-4070-8b57-26e3b32f94ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:51:34.510074 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 491/990] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:51:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 11:51:34.814797 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980a8cd4-b32c-4868-b47f-db085e9edf15 None None] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:51:34.815195 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980a8cd4-b32c-4868-b47f-db085e9edf15 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:51:34.815533 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980a8cd4-b32c-4868-b47f-db085e9edf15 None None] PATH_INFO: `/` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:51:34.816967 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 500/991] 10.222.0.59 () {54 vars in 964 bytes} [Tue Oct 11 11:51:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:51:34.823806 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05781cbc-33cf-4538-ad78-9493944c7fc3 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:51:34.824116 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05781cbc-33cf-4538-ad78-9493944c7fc3 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:51:34.824370 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05781cbc-33cf-4538-ad78-9493944c7fc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:51:34.839312 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:51:34.912261 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-05781cbc-33cf-4538-ad78-9493944c7fc3 None None] MFA Rules not processed for user `2e563fe174db4cc19ae6bf8162cc3d23`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:51:34.913587 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-05781cbc-33cf-4538-ad78-9493944c7fc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:51:34.997169 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 492/992] 10.222.0.59 () {58 vars in 1061 bytes} [Tue Oct 11 11:51:34 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 11:51:35.006051 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980a8cd4-b32c-4868-b47f-db085e9edf15 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:51:35.007680 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-980a8cd4-b32c-4868-b47f-db085e9edf15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:51:35.123669 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980a8cd4-b32c-4868-b47f-db085e9edf15 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:51:35.207573 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:51:35.210210 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7175c54-5abe-4154-8e0b-ef3df1dfa876 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e563fe174db4cc19ae6bf8162cc3d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:51:35.211080 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7175c54-5abe-4154-8e0b-ef3df1dfa876 service neutron] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:51:35.211389 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7175c54-5abe-4154-8e0b-ef3df1dfa876 service neutron] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:51:35.212057 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7175c54-5abe-4154-8e0b-ef3df1dfa876 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:51:35.214176 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-f7175c54-5abe-4154-8e0b-ef3df1dfa876 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:51:35.352393 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7175c54-5abe-4154-8e0b-ef3df1dfa876 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:51:35.353812 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7175c54-5abe-4154-8e0b-ef3df1dfa876 service neutron] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:51:35.376168 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 501/993] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 11:51:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 11:51:38.085003 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9733e5a-494a-4025-abe5-9887ba5a8792 None None] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:51:38.085552 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9733e5a-494a-4025-abe5-9887ba5a8792 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:51:38.085552 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9733e5a-494a-4025-abe5-9887ba5a8792 None None] PATH_INFO: `/` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:51:38.086944 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 493/994] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:51:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:51:38.092646 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6396f22-d18d-46db-bb5a-52720db1544f None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:51:38.093134 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6396f22-d18d-46db-bb5a-52720db1544f None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:51:38.093226 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6396f22-d18d-46db-bb5a-52720db1544f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:51:38.162809 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-f6396f22-d18d-46db-bb5a-52720db1544f None None] MFA Rules not processed for user `4965b07351a94e97bd57865e425127be`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:51:38.164040 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-f6396f22-d18d-46db-bb5a-52720db1544f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:51:38.220513 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:51:38.267548 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 502/995] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:51:38 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 11:51:38.552182 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9733e5a-494a-4025-abe5-9887ba5a8792 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:51:38.553736 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-f9733e5a-494a-4025-abe5-9887ba5a8792 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:51:38.667785 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9733e5a-494a-4025-abe5-9887ba5a8792 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:51:38.670863 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:51:38.746788 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6da8728-40ce-4738-9b14-c82f7c3fd8bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e563fe174db4cc19ae6bf8162cc3d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:51:38.747395 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6da8728-40ce-4738-9b14-c82f7c3fd8bc service neutron] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:51:38.747676 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6da8728-40ce-4738-9b14-c82f7c3fd8bc service neutron] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:51:38.747949 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6da8728-40ce-4738-9b14-c82f7c3fd8bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:51:38.749533 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-b6da8728-40ce-4738-9b14-c82f7c3fd8bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:51:38.861785 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6da8728-40ce-4738-9b14-c82f7c3fd8bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:51:38.863150 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6da8728-40ce-4738-9b14-c82f7c3fd8bc service neutron] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:51:38.881255 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 494/996] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 11:51:38 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 11:51:41.783498 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c25b7a-23e2-49d9-b01d-aa0bac684e11 None None] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:51:41.784056 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c25b7a-23e2-49d9-b01d-aa0bac684e11 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:51:41.784056 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c25b7a-23e2-49d9-b01d-aa0bac684e11 None None] PATH_INFO: `/` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:51:41.785402 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 503/997] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:51:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:51:41.790860 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fa0a3f-4ea9-4490-a01d-f4d451342d92 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:51:41.791318 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fa0a3f-4ea9-4490-a01d-f4d451342d92 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:51:41.791405 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fa0a3f-4ea9-4490-a01d-f4d451342d92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:51:41.832978 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:51:41.860357 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-10fa0a3f-4ea9-4490-a01d-f4d451342d92 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:51:41.861588 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-10fa0a3f-4ea9-4490-a01d-f4d451342d92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:51:41.965806 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 495/998] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:51:41 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 11:51:42.252141 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c25b7a-23e2-49d9-b01d-aa0bac684e11 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:51:42.254131 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-a3c25b7a-23e2-49d9-b01d-aa0bac684e11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:51:42.365338 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c25b7a-23e2-49d9-b01d-aa0bac684e11 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:51:42.444727 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b42283-e457-4695-a1b7-9e9b6dcea8b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e563fe174db4cc19ae6bf8162cc3d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:51:42.445377 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b42283-e457-4695-a1b7-9e9b6dcea8b8 service neutron] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:51:42.445647 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b42283-e457-4695-a1b7-9e9b6dcea8b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:51:42.445921 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b42283-e457-4695-a1b7-9e9b6dcea8b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:51:42.447641 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-d4b42283-e457-4695-a1b7-9e9b6dcea8b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:51:42.455037 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:51:42.563108 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b42283-e457-4695-a1b7-9e9b6dcea8b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:51:42.564661 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b42283-e457-4695-a1b7-9e9b6dcea8b8 service neutron] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:51:42.585135 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 504/999] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 11:51:42 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 11:51:45.516534 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bf102d-5701-4141-8b31-8709c1e6cc5d None None] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:51:45.517125 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bf102d-5701-4141-8b31-8709c1e6cc5d None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:51:45.517125 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bf102d-5701-4141-8b31-8709c1e6cc5d None None] PATH_INFO: `/` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:51:45.518488 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 496/1000] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:51:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:51:45.524582 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fc0eb2-c8e5-453f-af0d-717de35e141b None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:51:45.525140 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fc0eb2-c8e5-453f-af0d-717de35e141b None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:51:45.525140 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fc0eb2-c8e5-453f-af0d-717de35e141b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:51:45.599495 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-73fc0eb2-c8e5-453f-af0d-717de35e141b None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:51:45.600866 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-73fc0eb2-c8e5-453f-af0d-717de35e141b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:51:45.671933 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:51:45.722587 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 505/1001] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:51:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 11:51:46.007350 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bf102d-5701-4141-8b31-8709c1e6cc5d None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:51:46.009049 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-58bf102d-5701-4141-8b31-8709c1e6cc5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:51:46.132829 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bf102d-5701-4141-8b31-8709c1e6cc5d None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:51:46.147148 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:51:46.212142 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b43815-ebc0-40f1-a57b-f149ceb00834 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e563fe174db4cc19ae6bf8162cc3d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:51:46.212773 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b43815-ebc0-40f1-a57b-f149ceb00834 service neutron] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:51:46.213053 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b43815-ebc0-40f1-a57b-f149ceb00834 service neutron] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:51:46.213312 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b43815-ebc0-40f1-a57b-f149ceb00834 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:51:46.215009 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-76b43815-ebc0-40f1-a57b-f149ceb00834 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:51:46.330480 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b43815-ebc0-40f1-a57b-f149ceb00834 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:51:46.332197 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b43815-ebc0-40f1-a57b-f149ceb00834 service neutron] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:51:46.355914 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 497/1002] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 11:51:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 11:51:48.598076 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cef881-f888-4eb7-8c45-19a8ff35bc6e None None] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:51:48.598832 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cef881-f888-4eb7-8c45-19a8ff35bc6e None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:51:48.598832 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cef881-f888-4eb7-8c45-19a8ff35bc6e None None] PATH_INFO: `/` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:51:48.600065 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 506/1003] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:51:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:51:48.605711 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b1ef59-bf5e-4320-b9b5-b032c5b23110 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:51:48.606593 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b1ef59-bf5e-4320-b9b5-b032c5b23110 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:51:48.606913 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b1ef59-bf5e-4320-b9b5-b032c5b23110 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:51:48.660836 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:51:48.680946 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-c6b1ef59-bf5e-4320-b9b5-b032c5b23110 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:51:48.682154 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-c6b1ef59-bf5e-4320-b9b5-b032c5b23110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:51:48.793250 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 498/1004] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:51:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 11:51:49.072729 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cef881-f888-4eb7-8c45-19a8ff35bc6e None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:51:49.074289 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-83cef881-f888-4eb7-8c45-19a8ff35bc6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:51:49.201669 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cef881-f888-4eb7-8c45-19a8ff35bc6e None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:51:49.283473 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec6c99a-94db-4763-bc6e-60bdaf2d2e23 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e563fe174db4cc19ae6bf8162cc3d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:51:49.284086 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec6c99a-94db-4763-bc6e-60bdaf2d2e23 service neutron] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:51:49.284365 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec6c99a-94db-4763-bc6e-60bdaf2d2e23 service neutron] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:51:49.284618 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec6c99a-94db-4763-bc6e-60bdaf2d2e23 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:51:49.286232 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-eec6c99a-94db-4763-bc6e-60bdaf2d2e23 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:51:49.306296 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:51:49.404843 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec6c99a-94db-4763-bc6e-60bdaf2d2e23 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:51:49.406326 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec6c99a-94db-4763-bc6e-60bdaf2d2e23 service neutron] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:51:49.425826 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 507/1005] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 11:51:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 11:51:51.686464 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a172d1-3b1e-4f88-999c-48e19f9a4fe3 None None] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:51:51.687114 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a172d1-3b1e-4f88-999c-48e19f9a4fe3 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:51:51.687114 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a172d1-3b1e-4f88-999c-48e19f9a4fe3 None None] PATH_INFO: `/` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:51:51.688484 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 499/1006] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:51:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:51:51.694460 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665c880d-563d-439e-bbf3-3388b8cb6248 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:51:51.694975 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665c880d-563d-439e-bbf3-3388b8cb6248 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:51:51.695199 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665c880d-563d-439e-bbf3-3388b8cb6248 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:51:51.766897 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-665c880d-563d-439e-bbf3-3388b8cb6248 None None] MFA Rules not processed for user `4965b07351a94e97bd57865e425127be`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:51:51.768251 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-665c880d-563d-439e-bbf3-3388b8cb6248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:51:51.845918 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:51:51.883760 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 508/1007] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:51:51 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 11:51:52.161864 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a172d1-3b1e-4f88-999c-48e19f9a4fe3 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:51:52.163546 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-65a172d1-3b1e-4f88-999c-48e19f9a4fe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:51:52.293708 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a172d1-3b1e-4f88-999c-48e19f9a4fe3 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:51:52.321397 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:51:52.392809 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09587d71-563c-40e4-a4c4-52d5cf63553f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e563fe174db4cc19ae6bf8162cc3d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:51:52.393521 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09587d71-563c-40e4-a4c4-52d5cf63553f service neutron] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:51:52.393825 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09587d71-563c-40e4-a4c4-52d5cf63553f service neutron] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:51:52.394138 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09587d71-563c-40e4-a4c4-52d5cf63553f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:51:52.396174 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-09587d71-563c-40e4-a4c4-52d5cf63553f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:51:52.532537 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09587d71-563c-40e4-a4c4-52d5cf63553f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:51:52.534051 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09587d71-563c-40e4-a4c4-52d5cf63553f service neutron] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:51:52.557758 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 500/1008] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 11:51:52 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 11:51:55.028610 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6afba5d-8204-41f8-91ff-8ff38eaed516 None None] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:51:55.029224 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6afba5d-8204-41f8-91ff-8ff38eaed516 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:51:55.029224 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6afba5d-8204-41f8-91ff-8ff38eaed516 None None] PATH_INFO: `/` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:51:55.030602 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 509/1009] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:51:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 11 11:51:55.037166 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1de6903-1cda-46ba-8ab9-fbb01541bd41 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:51:55.037750 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1de6903-1cda-46ba-8ab9-fbb01541bd41 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:51:55.037962 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1de6903-1cda-46ba-8ab9-fbb01541bd41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:51:55.119143 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:51:55.128184 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-b1de6903-1cda-46ba-8ab9-fbb01541bd41 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:51:55.129523 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-b1de6903-1cda-46ba-8ab9-fbb01541bd41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:51:55.242861 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 501/1010] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:51:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 11:51:55.517653 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6afba5d-8204-41f8-91ff-8ff38eaed516 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:51:55.519266 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-a6afba5d-8204-41f8-91ff-8ff38eaed516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:51:55.628755 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6afba5d-8204-41f8-91ff-8ff38eaed516 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:51:55.706802 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c28333-e563-4e56-a3e3-c321e56a6dbc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e563fe174db4cc19ae6bf8162cc3d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:51:55.707410 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c28333-e563-4e56-a3e3-c321e56a6dbc service neutron] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:51:55.707683 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c28333-e563-4e56-a3e3-c321e56a6dbc service neutron] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:51:55.707945 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c28333-e563-4e56-a3e3-c321e56a6dbc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:51:55.709533 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-26c28333-e563-4e56-a3e3-c321e56a6dbc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:51:55.737667 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:51:55.826669 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c28333-e563-4e56-a3e3-c321e56a6dbc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:51:55.828122 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c28333-e563-4e56-a3e3-c321e56a6dbc service neutron] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:51:55.846586 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 510/1011] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 11:51:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 11:51:58.051871 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24db69e2-f65b-4fa7-ba56-2f837f103542 None None] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:51:58.052458 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24db69e2-f65b-4fa7-ba56-2f837f103542 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:51:58.052458 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24db69e2-f65b-4fa7-ba56-2f837f103542 None None] PATH_INFO: `/` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:51:58.053794 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 502/1012] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:51:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:51:58.059335 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d4adfb-88dc-4032-b81e-05088093f5dd None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:51:58.059832 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d4adfb-88dc-4032-b81e-05088093f5dd None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:51:58.059832 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d4adfb-88dc-4032-b81e-05088093f5dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:51:58.127872 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-62d4adfb-88dc-4032-b81e-05088093f5dd None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:51:58.129097 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-62d4adfb-88dc-4032-b81e-05088093f5dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:51:58.203446 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:51:58.226615 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 511/1013] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:51:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 11:51:58.503382 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24db69e2-f65b-4fa7-ba56-2f837f103542 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:51:58.505061 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-24db69e2-f65b-4fa7-ba56-2f837f103542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:51:58.612393 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24db69e2-f65b-4fa7-ba56-2f837f103542 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:51:58.642098 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:51:58.691516 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04947937-105c-498b-9a94-5c1d429f6f76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e563fe174db4cc19ae6bf8162cc3d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:51:58.692162 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04947937-105c-498b-9a94-5c1d429f6f76 service neutron] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:51:58.692471 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04947937-105c-498b-9a94-5c1d429f6f76 service neutron] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:51:58.692806 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04947937-105c-498b-9a94-5c1d429f6f76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:51:58.694514 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-04947937-105c-498b-9a94-5c1d429f6f76 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:51:58.817621 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04947937-105c-498b-9a94-5c1d429f6f76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:51:58.819135 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04947937-105c-498b-9a94-5c1d429f6f76 service neutron] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:51:58.839826 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 503/1014] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 11:51:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 11:52:01.347740 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9e466d-90f1-4b4f-9e4c-4455afe7e2a8 None None] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:52:01.348326 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9e466d-90f1-4b4f-9e4c-4455afe7e2a8 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:52:01.348326 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9e466d-90f1-4b4f-9e4c-4455afe7e2a8 None None] PATH_INFO: `/` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:52:01.349722 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 512/1015] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:52:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:52:01.355247 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5105a59d-1b52-48d4-989b-c7e781d392d3 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:52:01.355716 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5105a59d-1b52-48d4-989b-c7e781d392d3 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:52:01.355818 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5105a59d-1b52-48d4-989b-c7e781d392d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:52:01.436228 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-5105a59d-1b52-48d4-989b-c7e781d392d3 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:52:01.437473 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-5105a59d-1b52-48d4-989b-c7e781d392d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:52:01.446399 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:52:01.537854 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 504/1016] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:52:01 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 11:52:01.810499 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9e466d-90f1-4b4f-9e4c-4455afe7e2a8 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:52:01.812086 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-3e9e466d-90f1-4b4f-9e4c-4455afe7e2a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:52:01.921207 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9e466d-90f1-4b4f-9e4c-4455afe7e2a8 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:52:02.001944 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87479026-aabe-4c5a-89e3-bf1d9986e4bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e563fe174db4cc19ae6bf8162cc3d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:52:02.002586 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87479026-aabe-4c5a-89e3-bf1d9986e4bd service neutron] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:52:02.002871 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87479026-aabe-4c5a-89e3-bf1d9986e4bd service neutron] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:52:02.003143 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87479026-aabe-4c5a-89e3-bf1d9986e4bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:52:02.004811 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-87479026-aabe-4c5a-89e3-bf1d9986e4bd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:52:02.042185 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:52:02.119535 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87479026-aabe-4c5a-89e3-bf1d9986e4bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:52:02.120941 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87479026-aabe-4c5a-89e3-bf1d9986e4bd service neutron] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:52:02.140545 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 513/1017] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 11:52:01 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 11:52:06.465772 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0370d461-a673-4f8a-bc62-6e88fb5f5c1b None None] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:52:06.466333 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0370d461-a673-4f8a-bc62-6e88fb5f5c1b None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:52:06.466333 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0370d461-a673-4f8a-bc62-6e88fb5f5c1b None None] PATH_INFO: `/` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:52:06.468328 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 505/1018] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:52:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:52:06.473995 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2def660b-ab4e-4309-96f3-b1db54f3768f None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:52:06.474817 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2def660b-ab4e-4309-96f3-b1db54f3768f None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:52:06.474973 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2def660b-ab4e-4309-96f3-b1db54f3768f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:52:06.554444 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-2def660b-ab4e-4309-96f3-b1db54f3768f None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:52:06.555642 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-2def660b-ab4e-4309-96f3-b1db54f3768f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:52:06.658091 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:52:06.672197 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 514/1019] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:52:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 11:52:06.953873 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0370d461-a673-4f8a-bc62-6e88fb5f5c1b None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:52:06.955793 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-0370d461-a673-4f8a-bc62-6e88fb5f5c1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:52:07.075764 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0370d461-a673-4f8a-bc62-6e88fb5f5c1b None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:52:07.125897 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:52:07.166889 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0572ca95-d275-4862-bc73-27997a8b920d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e563fe174db4cc19ae6bf8162cc3d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:52:07.167510 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0572ca95-d275-4862-bc73-27997a8b920d service neutron] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:52:07.168151 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0572ca95-d275-4862-bc73-27997a8b920d service neutron] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:52:07.168151 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0572ca95-d275-4862-bc73-27997a8b920d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:52:07.169733 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-0572ca95-d275-4862-bc73-27997a8b920d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:52:07.289218 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0572ca95-d275-4862-bc73-27997a8b920d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:52:07.290582 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0572ca95-d275-4862-bc73-27997a8b920d service neutron] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:52:07.310403 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 506/1020] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 11:52:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 11:52:10.253504 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31197f9-5c63-4fef-86be-bd06c82df7f0 None None] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:52:10.254277 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31197f9-5c63-4fef-86be-bd06c82df7f0 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:52:10.254277 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31197f9-5c63-4fef-86be-bd06c82df7f0 None None] PATH_INFO: `/` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:52:10.256208 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 515/1021] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:52:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 11 11:52:10.264229 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486b51b5-ef82-4412-bb97-58b5b273f25b None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:52:10.265366 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486b51b5-ef82-4412-bb97-58b5b273f25b None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:52:10.265366 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486b51b5-ef82-4412-bb97-58b5b273f25b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:52:10.337360 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-486b51b5-ef82-4412-bb97-58b5b273f25b None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:52:10.338619 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-486b51b5-ef82-4412-bb97-58b5b273f25b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:52:10.358575 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:52:10.441635 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 507/1022] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:52:10 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 11:52:10.730620 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31197f9-5c63-4fef-86be-bd06c82df7f0 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:52:10.732195 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-b31197f9-5c63-4fef-86be-bd06c82df7f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:52:10.860296 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31197f9-5c63-4fef-86be-bd06c82df7f0 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:52:10.948010 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe49ed22-d756-4486-aede-af2e042dd969 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e563fe174db4cc19ae6bf8162cc3d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:52:10.948625 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe49ed22-d756-4486-aede-af2e042dd969 service neutron] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:52:10.948897 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe49ed22-d756-4486-aede-af2e042dd969 service neutron] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:52:10.949162 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe49ed22-d756-4486-aede-af2e042dd969 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:52:10.950791 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-fe49ed22-d756-4486-aede-af2e042dd969 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:52:10.999302 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:52:11.070787 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe49ed22-d756-4486-aede-af2e042dd969 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:52:11.072196 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe49ed22-d756-4486-aede-af2e042dd969 service neutron] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:52:11.094322 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 516/1023] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 11:52:10 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 11:52:15.457771 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eef323-c566-4473-a442-30a5152b1dab None None] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:52:15.458817 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eef323-c566-4473-a442-30a5152b1dab None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:52:15.458817 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eef323-c566-4473-a442-30a5152b1dab None None] PATH_INFO: `/` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:52:15.460018 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 508/1024] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:52:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 11 11:52:15.471736 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5b054f-fe82-4368-9492-d918ca47dac1 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:52:15.472227 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5b054f-fe82-4368-9492-d918ca47dac1 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:52:15.472702 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5b054f-fe82-4368-9492-d918ca47dac1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:52:15.576226 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-5d5b054f-fe82-4368-9492-d918ca47dac1 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:52:15.577536 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-5d5b054f-fe82-4368-9492-d918ca47dac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:52:15.693085 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:52:15.695982 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 517/1025] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:52:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 11:52:15.981419 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eef323-c566-4473-a442-30a5152b1dab None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:52:15.983086 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-83eef323-c566-4473-a442-30a5152b1dab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:52:16.096562 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eef323-c566-4473-a442-30a5152b1dab None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:52:16.149821 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:52:16.178324 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865b95d7-21e2-4969-906b-66b084fd5f84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e563fe174db4cc19ae6bf8162cc3d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:52:16.178978 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865b95d7-21e2-4969-906b-66b084fd5f84 service neutron] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:52:16.179260 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865b95d7-21e2-4969-906b-66b084fd5f84 service neutron] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:52:16.179538 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865b95d7-21e2-4969-906b-66b084fd5f84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:52:16.181200 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-865b95d7-21e2-4969-906b-66b084fd5f84 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:52:16.305441 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865b95d7-21e2-4969-906b-66b084fd5f84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:52:16.308232 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865b95d7-21e2-4969-906b-66b084fd5f84 service neutron] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:52:16.327211 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 509/1026] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 11:52:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 11:52:21.472295 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c437048-27f4-4659-9878-f6e10f79f7ab None None] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:52:21.472295 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c437048-27f4-4659-9878-f6e10f79f7ab None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:52:21.472295 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c437048-27f4-4659-9878-f6e10f79f7ab None None] PATH_INFO: `/` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:52:21.475292 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 518/1027] 10.222.0.59 () {54 vars in 921 bytes} [Tue Oct 11 11:52:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:52:21.481450 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaa5b59-440f-4826-857b-a0657e7dfdc3 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:52:21.482383 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaa5b59-440f-4826-857b-a0657e7dfdc3 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:52:21.482874 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaa5b59-440f-4826-857b-a0657e7dfdc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:52:21.610253 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-9eaa5b59-440f-4826-857b-a0657e7dfdc3 None None] MFA Rules not processed for user `f272cdd1bcc04b4e8261ed533cfd4fd7`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:52:21.611650 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-9eaa5b59-440f-4826-857b-a0657e7dfdc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:52:21.642949 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:52:21.759131 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 510/1028] 10.222.0.59 () {58 vars in 1018 bytes} [Tue Oct 11 11:52:21 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 11:52:27.153533 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb88517a-1288-4f84-90c7-ce3c451ba530 None None] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:52:27.153533 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb88517a-1288-4f84-90c7-ce3c451ba530 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:52:27.153533 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb88517a-1288-4f84-90c7-ce3c451ba530 None None] PATH_INFO: `/` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:52:27.154852 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 519/1029] 10.222.0.59 () {54 vars in 921 bytes} [Tue Oct 11 11:52:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:52:27.161841 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21581fa-487a-4169-b1f2-600827a30074 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:52:27.162527 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21581fa-487a-4169-b1f2-600827a30074 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:52:27.163224 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21581fa-487a-4169-b1f2-600827a30074 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:52:27.322470 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-c21581fa-487a-4169-b1f2-600827a30074 None None] MFA Rules not processed for user `f272cdd1bcc04b4e8261ed533cfd4fd7`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:52:27.324784 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-c21581fa-487a-4169-b1f2-600827a30074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:52:27.453761 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:52:27.521058 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 511/1030] 10.222.0.59 () {58 vars in 1018 bytes} [Tue Oct 11 11:52:27 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 361 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 11:52:30.584841 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350c40d6-28ea-4a76-88ed-50520696ac97 None None] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:52:30.585591 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350c40d6-28ea-4a76-88ed-50520696ac97 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:52:30.585591 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350c40d6-28ea-4a76-88ed-50520696ac97 None None] PATH_INFO: `/` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:52:30.587281 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 520/1031] 10.222.0.59 () {54 vars in 921 bytes} [Tue Oct 11 11:52:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:52:30.594260 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f59557-6b54-4047-94ad-a115652aadc9 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:52:30.595135 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f59557-6b54-4047-94ad-a115652aadc9 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:52:30.595440 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f59557-6b54-4047-94ad-a115652aadc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:52:30.746439 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-11f59557-6b54-4047-94ad-a115652aadc9 None None] MFA Rules not processed for user `f272cdd1bcc04b4e8261ed533cfd4fd7`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:52:30.747704 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-11f59557-6b54-4047-94ad-a115652aadc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:52:30.886219 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 512/1032] 10.222.0.59 () {58 vars in 1018 bytes} [Tue Oct 11 11:52:30 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 11:52:32.903250 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ae6ea5-9df1-45dd-987a-d738e356b3cb None None] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:52:32.904091 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ae6ea5-9df1-45dd-987a-d738e356b3cb None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:52:32.904091 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ae6ea5-9df1-45dd-987a-d738e356b3cb None None] PATH_INFO: `/` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:52:32.905167 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 521/1033] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:52:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:52:32.911851 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59232440-24f2-44cd-b63f-7ba6276a341b None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:52:32.912476 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59232440-24f2-44cd-b63f-7ba6276a341b None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:52:32.912476 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59232440-24f2-44cd-b63f-7ba6276a341b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:52:32.973453 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:52:33.008159 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-59232440-24f2-44cd-b63f-7ba6276a341b None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:52:33.009590 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-59232440-24f2-44cd-b63f-7ba6276a341b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:52:33.207133 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 513/1034] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:52:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 11:52:33.516015 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b0a2af-24a3-4aee-9e56-311c6c8c99fe None None] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:52:33.516472 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b0a2af-24a3-4aee-9e56-311c6c8c99fe None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:52:33.516924 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b0a2af-24a3-4aee-9e56-311c6c8c99fe None None] PATH_INFO: `/` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:52:33.520759 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 522/1035] 10.222.0.59 () {54 vars in 961 bytes} [Tue Oct 11 11:52:33 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:52:33.531158 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42836607-4e2d-4d17-8b37-b1255c6175d5 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:52:33.532098 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42836607-4e2d-4d17-8b37-b1255c6175d5 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:52:33.532648 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42836607-4e2d-4d17-8b37-b1255c6175d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:52:33.676510 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-42836607-4e2d-4d17-8b37-b1255c6175d5 None None] MFA Rules not processed for user `bad5e1cb306b46818096940acc3810b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:52:33.678445 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-42836607-4e2d-4d17-8b37-b1255c6175d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:52:33.767599 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:52:33.890921 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 514/1036] 10.222.0.59 () {58 vars in 1058 bytes} [Tue Oct 11 11:52:33 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 363 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 11:52:33.903706 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b0a2af-24a3-4aee-9e56-311c6c8c99fe None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:52:33.905757 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-51b0a2af-24a3-4aee-9e56-311c6c8c99fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:52:34.115071 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b0a2af-24a3-4aee-9e56-311c6c8c99fe None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:52:34.219693 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:52:34.306133 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de7c833-2d57-46fa-a934-35bfb6ac9950 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bad5e1cb306b46818096940acc3810b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:52:34.308016 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de7c833-2d57-46fa-a934-35bfb6ac9950 service nova] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:52:34.308391 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de7c833-2d57-46fa-a934-35bfb6ac9950 service nova] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:52:34.308639 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de7c833-2d57-46fa-a934-35bfb6ac9950 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:52:34.310455 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-1de7c833-2d57-46fa-a934-35bfb6ac9950 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:52:34.421098 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de7c833-2d57-46fa-a934-35bfb6ac9950 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:52:34.422633 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de7c833-2d57-46fa-a934-35bfb6ac9950 service nova] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:52:34.443243 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 523/1037] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 11:52:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 540 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 11:52:37.057903 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e25c84-2018-48fb-a1ce-116971fa2d48 None None] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:52:37.058455 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e25c84-2018-48fb-a1ce-116971fa2d48 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:52:37.058736 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e25c84-2018-48fb-a1ce-116971fa2d48 None None] PATH_INFO: `/` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:52:37.061447 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 515/1038] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:52:37 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:52:37.068052 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6329cd-3695-440d-a0f7-2cf9be3cea4e None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:52:37.068880 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6329cd-3695-440d-a0f7-2cf9be3cea4e None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:52:37.069345 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6329cd-3695-440d-a0f7-2cf9be3cea4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:52:37.178863 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-2d6329cd-3695-440d-a0f7-2cf9be3cea4e None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:52:37.180515 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-2d6329cd-3695-440d-a0f7-2cf9be3cea4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:52:37.237839 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:52:37.326696 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 524/1039] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:52:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 11:52:37.628539 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e25c84-2018-48fb-a1ce-116971fa2d48 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:52:37.632528 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-a8e25c84-2018-48fb-a1ce-116971fa2d48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:52:37.923835 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e25c84-2018-48fb-a1ce-116971fa2d48 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:52:37.934369 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:52:38.081324 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c5cd91-013b-49ce-abbd-b4e874ec49d4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bad5e1cb306b46818096940acc3810b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:52:38.082043 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c5cd91-013b-49ce-abbd-b4e874ec49d4 service nova] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:52:38.082275 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c5cd91-013b-49ce-abbd-b4e874ec49d4 service nova] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:52:38.082550 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c5cd91-013b-49ce-abbd-b4e874ec49d4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:52:38.084402 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-07c5cd91-013b-49ce-abbd-b4e874ec49d4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:52:38.202006 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c5cd91-013b-49ce-abbd-b4e874ec49d4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:52:38.204950 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c5cd91-013b-49ce-abbd-b4e874ec49d4 service nova] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:52:38.230464 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 516/1040] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 11:52:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 604 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 11 11:52:40.795281 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e50e4f3-d022-4893-b5b2-5c66be41dca7 None None] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:52:40.795945 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e50e4f3-d022-4893-b5b2-5c66be41dca7 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:52:40.795945 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e50e4f3-d022-4893-b5b2-5c66be41dca7 None None] PATH_INFO: `/` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:52:40.797280 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 525/1041] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:52:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:52:40.803775 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f72b32-cc80-422e-9d1e-3960b2c1eb25 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:52:40.804374 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f72b32-cc80-422e-9d1e-3960b2c1eb25 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:52:40.804478 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f72b32-cc80-422e-9d1e-3960b2c1eb25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:52:40.836828 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:52:40.917787 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-96f72b32-cc80-422e-9d1e-3960b2c1eb25 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:52:40.919091 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-96f72b32-cc80-422e-9d1e-3960b2c1eb25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:52:41.050706 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 517/1042] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:52:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 11:52:41.344767 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbd2913-b18c-48dd-976f-8eb051cb4b73 None None] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:52:41.345315 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbd2913-b18c-48dd-976f-8eb051cb4b73 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:52:41.345717 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbd2913-b18c-48dd-976f-8eb051cb4b73 None None] PATH_INFO: `/` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:52:41.347540 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 526/1043] 10.222.0.59 () {54 vars in 961 bytes} [Tue Oct 11 11:52:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 11 11:52:41.353404 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d88d87-f551-4f27-9d51-56d5b82777a2 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:52:41.353833 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d88d87-f551-4f27-9d51-56d5b82777a2 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:52:41.354210 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d88d87-f551-4f27-9d51-56d5b82777a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:52:41.444802 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-80d88d87-f551-4f27-9d51-56d5b82777a2 None None] MFA Rules not processed for user `bad5e1cb306b46818096940acc3810b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:52:41.446194 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-80d88d87-f551-4f27-9d51-56d5b82777a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:52:41.487145 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:52:41.591220 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 518/1044] 10.222.0.59 () {58 vars in 1058 bytes} [Tue Oct 11 11:52:41 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 11:52:41.600240 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbd2913-b18c-48dd-976f-8eb051cb4b73 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:52:41.602430 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-1cbd2913-b18c-48dd-976f-8eb051cb4b73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:52:41.836180 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbd2913-b18c-48dd-976f-8eb051cb4b73 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:52:41.869586 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:52:42.043101 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecf4014-105a-416d-9561-ff75533f7bd8 None None] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:52:42.044061 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecf4014-105a-416d-9561-ff75533f7bd8 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:52:42.044280 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecf4014-105a-416d-9561-ff75533f7bd8 None None] PATH_INFO: `/` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:52:42.047323 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 519/1045] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:52:42 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:52:42.053785 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aaa17d-0847-4f33-a863-d3cc3de68e61 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:52:42.054048 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aaa17d-0847-4f33-a863-d3cc3de68e61 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:52:42.054484 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aaa17d-0847-4f33-a863-d3cc3de68e61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:52:42.070707 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048609e4-4c9a-431a-8639-57a14495cbc3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bad5e1cb306b46818096940acc3810b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:52:42.071347 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048609e4-4c9a-431a-8639-57a14495cbc3 service nova] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:52:42.072230 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048609e4-4c9a-431a-8639-57a14495cbc3 service nova] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:52:42.072514 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048609e4-4c9a-431a-8639-57a14495cbc3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:52:42.074310 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-048609e4-4c9a-431a-8639-57a14495cbc3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:52:42.179032 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-a4aaa17d-0847-4f33-a863-d3cc3de68e61 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:52:42.181310 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-a4aaa17d-0847-4f33-a863-d3cc3de68e61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:52:42.183075 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048609e4-4c9a-431a-8639-57a14495cbc3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:52:42.184757 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048609e4-4c9a-431a-8639-57a14495cbc3 service nova] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:52:42.205732 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 527/1046] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 11:52:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 606 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 11 11:52:42.325043 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:52:42.337560 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 520/1047] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:52:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 11:52:42.638793 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4328fc04-c8c2-4a90-87a1-0bedd195889b None None] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:52:42.639225 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4328fc04-c8c2-4a90-87a1-0bedd195889b None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:52:42.639591 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4328fc04-c8c2-4a90-87a1-0bedd195889b None None] PATH_INFO: `/` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:52:42.641252 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 528/1048] 10.222.0.59 () {54 vars in 963 bytes} [Tue Oct 11 11:52:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:52:42.646601 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eb6d90-3342-4047-8189-84199983d6ea None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:52:42.647019 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eb6d90-3342-4047-8189-84199983d6ea None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:52:42.647523 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eb6d90-3342-4047-8189-84199983d6ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:52:42.739388 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-03eb6d90-3342-4047-8189-84199983d6ea None None] MFA Rules not processed for user `2b1b651c73664d08accc6edf2f79bace`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:52:42.740715 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-03eb6d90-3342-4047-8189-84199983d6ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:52:42.859568 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 521/1049] 10.222.0.59 () {58 vars in 1060 bytes} [Tue Oct 11 11:52:42 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 11:52:42.872877 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4328fc04-c8c2-4a90-87a1-0bedd195889b None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:52:42.875595 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-4328fc04-c8c2-4a90-87a1-0bedd195889b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:52:42.934290 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:52:43.118098 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4328fc04-c8c2-4a90-87a1-0bedd195889b None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:52:43.281600 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb43cb0-7d68-4f96-b8b7-bfbf9bbdc723 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b1b651c73664d08accc6edf2f79bace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:52:43.282235 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb43cb0-7d68-4f96-b8b7-bfbf9bbdc723 service cinder] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:52:43.282521 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb43cb0-7d68-4f96-b8b7-bfbf9bbdc723 service cinder] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:52:43.282852 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb43cb0-7d68-4f96-b8b7-bfbf9bbdc723 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:52:43.284801 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-dcb43cb0-7d68-4f96-b8b7-bfbf9bbdc723 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:52:43.370967 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:52:43.409702 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb43cb0-7d68-4f96-b8b7-bfbf9bbdc723 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:52:43.411232 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb43cb0-7d68-4f96-b8b7-bfbf9bbdc723 service cinder] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:52:43.430200 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 529/1050] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 11:52:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 558 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 11:52:44.759929 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fe695c-1141-4cb3-8c46-166332435974 None None] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:52:44.760519 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fe695c-1141-4cb3-8c46-166332435974 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:52:44.760519 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fe695c-1141-4cb3-8c46-166332435974 None None] PATH_INFO: `/` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:52:44.762471 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 522/1051] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:52:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:52:44.768540 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcf343e-b703-4959-8d20-4451c9206f35 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:52:44.769126 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcf343e-b703-4959-8d20-4451c9206f35 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:52:44.769126 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcf343e-b703-4959-8d20-4451c9206f35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:52:44.850105 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-4dcf343e-b703-4959-8d20-4451c9206f35 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:52:44.851560 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-4dcf343e-b703-4959-8d20-4451c9206f35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:52:44.970263 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 530/1052] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:52:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 11:52:45.259869 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fe695c-1141-4cb3-8c46-166332435974 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:52:45.262668 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-05fe695c-1141-4cb3-8c46-166332435974 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:52:45.423288 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:52:45.519977 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fe695c-1141-4cb3-8c46-166332435974 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:52:45.657968 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d2a27f-1ce0-47e6-b915-f328acda899b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bad5e1cb306b46818096940acc3810b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:52:45.658898 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d2a27f-1ce0-47e6-b915-f328acda899b service nova] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:52:45.659223 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d2a27f-1ce0-47e6-b915-f328acda899b service nova] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:52:45.659494 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d2a27f-1ce0-47e6-b915-f328acda899b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:52:45.661277 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-33d2a27f-1ce0-47e6-b915-f328acda899b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:52:45.732732 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:52:45.786030 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d2a27f-1ce0-47e6-b915-f328acda899b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:52:45.787994 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d2a27f-1ce0-47e6-b915-f328acda899b service nova] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:52:45.809342 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 523/1053] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 11:52:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 551 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 11:52:48.127486 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa847191-4f04-44a4-b61e-57feb379beb9 None None] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:52:48.128146 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa847191-4f04-44a4-b61e-57feb379beb9 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:52:48.128146 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa847191-4f04-44a4-b61e-57feb379beb9 None None] PATH_INFO: `/` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:52:48.130602 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 531/1054] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:52:48 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:52:48.136118 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4165fcc2-92fe-4957-a92b-8a5ed7693836 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:52:48.136663 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4165fcc2-92fe-4957-a92b-8a5ed7693836 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:52:48.136663 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4165fcc2-92fe-4957-a92b-8a5ed7693836 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:52:48.217518 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-4165fcc2-92fe-4957-a92b-8a5ed7693836 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:52:48.218687 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-4165fcc2-92fe-4957-a92b-8a5ed7693836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:52:48.356599 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 524/1055] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:52:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 11:52:48.651806 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa847191-4f04-44a4-b61e-57feb379beb9 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:52:48.653645 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-aa847191-4f04-44a4-b61e-57feb379beb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:52:48.694902 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:52:48.816809 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa847191-4f04-44a4-b61e-57feb379beb9 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:52:48.974950 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:52:48.982079 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01169f17-98e9-4a9c-a33c-b42df33bbb2e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bad5e1cb306b46818096940acc3810b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:52:48.982974 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01169f17-98e9-4a9c-a33c-b42df33bbb2e service nova] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:52:48.983857 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01169f17-98e9-4a9c-a33c-b42df33bbb2e service nova] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:52:48.983857 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01169f17-98e9-4a9c-a33c-b42df33bbb2e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:52:48.985343 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-01169f17-98e9-4a9c-a33c-b42df33bbb2e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:52:49.141754 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01169f17-98e9-4a9c-a33c-b42df33bbb2e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:52:49.143630 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01169f17-98e9-4a9c-a33c-b42df33bbb2e service nova] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:52:49.164998 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 532/1056] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 11:52:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 514 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 11:52:51.444089 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e457fa-5d23-4c83-bf70-e044b4a370b6 None None] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:52:51.444684 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e457fa-5d23-4c83-bf70-e044b4a370b6 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:52:51.444894 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e457fa-5d23-4c83-bf70-e044b4a370b6 None None] PATH_INFO: `/` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:52:51.449999 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 525/1057] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:52:51 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:52:51.455358 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60968ac-e9a1-4d32-8408-d1205422632f None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:52:51.456321 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60968ac-e9a1-4d32-8408-d1205422632f None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:52:51.456585 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60968ac-e9a1-4d32-8408-d1205422632f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:52:51.551669 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-c60968ac-e9a1-4d32-8408-d1205422632f None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:52:51.553549 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-c60968ac-e9a1-4d32-8408-d1205422632f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:52:51.654365 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:52:51.702002 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 533/1058] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:52:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 11:52:52.109446 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e457fa-5d23-4c83-bf70-e044b4a370b6 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:52:52.111142 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-70e457fa-5d23-4c83-bf70-e044b4a370b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:52:52.157094 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:52:52.340806 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e457fa-5d23-4c83-bf70-e044b4a370b6 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:52:52.509423 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:52:52.537332 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108eb9e8-df79-4c89-b9e4-535b684956c0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bad5e1cb306b46818096940acc3810b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:52:52.537982 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108eb9e8-df79-4c89-b9e4-535b684956c0 service nova] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:52:52.538263 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108eb9e8-df79-4c89-b9e4-535b684956c0 service nova] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:52:52.538530 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108eb9e8-df79-4c89-b9e4-535b684956c0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:52:52.540336 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-108eb9e8-df79-4c89-b9e4-535b684956c0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:52:52.660802 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108eb9e8-df79-4c89-b9e4-535b684956c0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:52:52.662620 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108eb9e8-df79-4c89-b9e4-535b684956c0 service nova] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:52:52.685718 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 526/1059] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 11:52:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 577 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 11:52:55.026576 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1455437-4473-4b0b-aa10-b641797d5e9b None None] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:52:55.027153 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1455437-4473-4b0b-aa10-b641797d5e9b None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:52:55.027153 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1455437-4473-4b0b-aa10-b641797d5e9b None None] PATH_INFO: `/` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:52:55.029049 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 534/1060] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:52:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:52:55.035321 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d13d90-bbf4-4999-8e3b-717aad45a763 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:52:55.035840 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d13d90-bbf4-4999-8e3b-717aad45a763 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:52:55.035944 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d13d90-bbf4-4999-8e3b-717aad45a763 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:52:55.120263 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-31d13d90-bbf4-4999-8e3b-717aad45a763 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:52:55.121574 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-31d13d90-bbf4-4999-8e3b-717aad45a763 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:52:55.204703 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:52:55.257713 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 527/1061] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:52:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 11:52:55.553296 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1455437-4473-4b0b-aa10-b641797d5e9b None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:52:55.555001 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-b1455437-4473-4b0b-aa10-b641797d5e9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:52:55.735867 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1455437-4473-4b0b-aa10-b641797d5e9b None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:52:55.803182 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:52:55.883668 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665f88f3-6aba-46bd-bfe6-6ef7c6a284ca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bad5e1cb306b46818096940acc3810b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:52:55.884355 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665f88f3-6aba-46bd-bfe6-6ef7c6a284ca service nova] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:52:55.884649 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665f88f3-6aba-46bd-bfe6-6ef7c6a284ca service nova] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:52:55.884957 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665f88f3-6aba-46bd-bfe6-6ef7c6a284ca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:52:55.886678 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-665f88f3-6aba-46bd-bfe6-6ef7c6a284ca service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:52:55.988278 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665f88f3-6aba-46bd-bfe6-6ef7c6a284ca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:52:55.990011 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665f88f3-6aba-46bd-bfe6-6ef7c6a284ca service nova] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:52:56.010186 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 535/1062] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 11:52:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 11:52:58.241265 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0fa5e8-7d47-4158-a619-76df2c131272 None None] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:52:58.241806 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0fa5e8-7d47-4158-a619-76df2c131272 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:52:58.241806 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0fa5e8-7d47-4158-a619-76df2c131272 None None] PATH_INFO: `/` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:52:58.243384 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 528/1063] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:52:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:52:58.249503 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27128e5-3e13-4c6b-8651-c59782e578e0 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:52:58.250120 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27128e5-3e13-4c6b-8651-c59782e578e0 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:52:58.250120 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27128e5-3e13-4c6b-8651-c59782e578e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:52:58.337947 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-e27128e5-3e13-4c6b-8651-c59782e578e0 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:52:58.339303 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-e27128e5-3e13-4c6b-8651-c59782e578e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:52:58.349516 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:52:58.497532 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 536/1064] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:52:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 11:52:58.792844 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0fa5e8-7d47-4158-a619-76df2c131272 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:52:58.794607 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-2e0fa5e8-7d47-4158-a619-76df2c131272 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:52:58.971395 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0fa5e8-7d47-4158-a619-76df2c131272 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:52:59.027106 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:52:59.106819 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e724df-3bc4-4280-a002-1c4f77bc8777 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bad5e1cb306b46818096940acc3810b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:52:59.107497 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e724df-3bc4-4280-a002-1c4f77bc8777 service nova] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:52:59.107926 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e724df-3bc4-4280-a002-1c4f77bc8777 service nova] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:52:59.108213 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e724df-3bc4-4280-a002-1c4f77bc8777 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:52:59.109940 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-95e724df-3bc4-4280-a002-1c4f77bc8777 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:52:59.221243 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e724df-3bc4-4280-a002-1c4f77bc8777 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:52:59.222837 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e724df-3bc4-4280-a002-1c4f77bc8777 service nova] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:52:59.243674 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 529/1065] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 11:52:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 11:53:00.422196 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b47869-5c21-460c-b935-49013b5c5471 None None] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:53:00.422845 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b47869-5c21-460c-b935-49013b5c5471 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:53:00.422845 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b47869-5c21-460c-b935-49013b5c5471 None None] PATH_INFO: `/` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:53:00.424253 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 537/1066] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:53:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:53:00.430260 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c990f36-1e25-4976-9fff-f92463e876b8 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:53:00.430923 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c990f36-1e25-4976-9fff-f92463e876b8 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:53:00.430923 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c990f36-1e25-4976-9fff-f92463e876b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:53:00.499820 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:53:00.503535 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-6c990f36-1e25-4976-9fff-f92463e876b8 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:53:00.505266 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-6c990f36-1e25-4976-9fff-f92463e876b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:53:00.615186 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 530/1067] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:53:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 11:53:00.915782 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2445205-2252-4494-b016-3421edbe3e73 None None] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:53:00.916059 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2445205-2252-4494-b016-3421edbe3e73 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:53:00.916351 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2445205-2252-4494-b016-3421edbe3e73 None None] PATH_INFO: `/` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:53:00.917901 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 538/1068] 10.222.0.59 () {54 vars in 962 bytes} [Tue Oct 11 11:53:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:53:00.926000 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18b249a-675a-40c0-9322-fb835a310207 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:53:00.926617 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18b249a-675a-40c0-9322-fb835a310207 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:53:00.927191 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18b249a-675a-40c0-9322-fb835a310207 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:53:01.025881 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-e18b249a-675a-40c0-9322-fb835a310207 None None] MFA Rules not processed for user `91a00658b79a42bd91a22a2508b5de34`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:53:01.027483 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-e18b249a-675a-40c0-9322-fb835a310207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:53:01.108206 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:53:01.124440 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 531/1069] 10.222.0.59 () {58 vars in 1059 bytes} [Tue Oct 11 11:53:00 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 11:53:01.135048 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2445205-2252-4494-b016-3421edbe3e73 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:53:01.138301 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-e2445205-2252-4494-b016-3421edbe3e73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:53:01.281347 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2445205-2252-4494-b016-3421edbe3e73 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:53:01.325642 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:53:01.367994 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f5c9d0-664d-4476-a1c8-b20a42fd1dfb service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91a00658b79a42bd91a22a2508b5de34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:53:01.368698 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f5c9d0-664d-4476-a1c8-b20a42fd1dfb service glance] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:53:01.368978 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f5c9d0-664d-4476-a1c8-b20a42fd1dfb service glance] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:53:01.369251 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f5c9d0-664d-4476-a1c8-b20a42fd1dfb service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:53:01.370965 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-88f5c9d0-664d-4476-a1c8-b20a42fd1dfb service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:53:01.498838 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f5c9d0-664d-4476-a1c8-b20a42fd1dfb service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:53:01.500222 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f5c9d0-664d-4476-a1c8-b20a42fd1dfb service glance] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:53:01.523964 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 539/1070] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 11:53:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 11:53:01.646930 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6bfa7c-49c2-4f93-a483-9833d3846d62 None None] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:53:01.647732 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6bfa7c-49c2-4f93-a483-9833d3846d62 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:53:01.647732 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6bfa7c-49c2-4f93-a483-9833d3846d62 None None] PATH_INFO: `/` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:53:01.648997 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 532/1071] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:53:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:53:01.662930 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30300e6d-f884-47fc-a64f-48b816031520 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:53:01.663250 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30300e6d-f884-47fc-a64f-48b816031520 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:53:01.663534 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30300e6d-f884-47fc-a64f-48b816031520 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:53:01.742890 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-30300e6d-f884-47fc-a64f-48b816031520 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:53:01.744211 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-30300e6d-f884-47fc-a64f-48b816031520 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:53:01.768797 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:53:01.879830 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 540/1072] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:53:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 11:53:02.184673 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6bfa7c-49c2-4f93-a483-9833d3846d62 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:53:02.186612 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-4a6bfa7c-49c2-4f93-a483-9833d3846d62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:53:02.358527 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd432564-45ba-4ea7-90e4-08aef0c362ef None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:53:02.358973 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd432564-45ba-4ea7-90e4-08aef0c362ef None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:53:02.359325 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd432564-45ba-4ea7-90e4-08aef0c362ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:53:02.384347 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6bfa7c-49c2-4f93-a483-9833d3846d62 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:53:02.438379 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-fd432564-45ba-4ea7-90e4-08aef0c362ef None None] MFA Rules not processed for user `8a19eceb62ce4157a4c5fe1866a89df9`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:53:02.439639 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-fd432564-45ba-4ea7-90e4-08aef0c362ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:53:02.461287 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:53:02.513405 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b337acf9-d220-47a9-b2ac-12dacf3b28ac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bad5e1cb306b46818096940acc3810b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:53:02.514058 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b337acf9-d220-47a9-b2ac-12dacf3b28ac service nova] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:53:02.514346 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b337acf9-d220-47a9-b2ac-12dacf3b28ac service nova] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:53:02.514650 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b337acf9-d220-47a9-b2ac-12dacf3b28ac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:53:02.516380 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-b337acf9-d220-47a9-b2ac-12dacf3b28ac service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:53:02.535616 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 541/1073] 10.222.0.59 () {58 vars in 1009 bytes} [Tue Oct 11 11:53:02 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 11:53:02.588025 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb81ac0-b0c8-4e24-9903-691478bfb00e None None] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:53:02.588025 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb81ac0-b0c8-4e24-9903-691478bfb00e None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:53:02.588025 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb81ac0-b0c8-4e24-9903-691478bfb00e None None] PATH_INFO: `/` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:53:02.589313 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 542/1074] 10.222.0.59 () {54 vars in 943 bytes} [Tue Oct 11 11:53:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:53:02.597642 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997fbf77-8d98-4e38-8106-e5a17d0c8efb None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:53:02.597642 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997fbf77-8d98-4e38-8106-e5a17d0c8efb None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:53:02.597642 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997fbf77-8d98-4e38-8106-e5a17d0c8efb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:53:02.615063 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:53:02.638735 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b337acf9-d220-47a9-b2ac-12dacf3b28ac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:53:02.640385 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b337acf9-d220-47a9-b2ac-12dacf3b28ac service nova] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:53:02.659787 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 533/1075] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 11:53:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 11:53:02.691778 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-997fbf77-8d98-4e38-8106-e5a17d0c8efb None None] MFA Rules not processed for user `76e7fee1993a40d5ba4295e7daeb3e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:53:02.693154 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-997fbf77-8d98-4e38-8106-e5a17d0c8efb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:53:02.825955 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 543/1076] 10.222.0.59 () {58 vars in 1040 bytes} [Tue Oct 11 11:53:02 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 11:53:02.838323 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b337acf9-d220-47a9-b2ac-12dacf3b28ac service nova] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:53:02.839685 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-b337acf9-d220-47a9-b2ac-12dacf3b28ac service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:53:02.936516 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:53:02.941290 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b337acf9-d220-47a9-b2ac-12dacf3b28ac service nova] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:53:03.060446 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85085af7-e10b-4832-98db-f5803de8e0e3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76e7fee1993a40d5ba4295e7daeb3e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:53:03.061067 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85085af7-e10b-4832-98db-f5803de8e0e3 service swift] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:53:03.061337 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85085af7-e10b-4832-98db-f5803de8e0e3 service swift] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:53:03.061601 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85085af7-e10b-4832-98db-f5803de8e0e3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:53:03.063282 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-85085af7-e10b-4832-98db-f5803de8e0e3 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:53:03.159630 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85085af7-e10b-4832-98db-f5803de8e0e3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:53:03.161753 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85085af7-e10b-4832-98db-f5803de8e0e3 service swift] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:53:03.172321 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 534/1077] 10.222.0.59 () {60 vars in 1384 bytes} [Tue Oct 11 11:53:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 11 11:53:04.910184 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28ea5f4-4a3a-4588-8cf9-95d3f7e71a2a None None] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:53:04.910816 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28ea5f4-4a3a-4588-8cf9-95d3f7e71a2a None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:53:04.910816 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28ea5f4-4a3a-4588-8cf9-95d3f7e71a2a None None] PATH_INFO: `/` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:53:04.912212 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 544/1078] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:53:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:53:04.917930 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde40eb5-dde8-474e-92db-d73d31ddc4fa None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:53:04.918442 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde40eb5-dde8-474e-92db-d73d31ddc4fa None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:53:04.918523 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde40eb5-dde8-474e-92db-d73d31ddc4fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:53:04.974927 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:53:05.022533 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-bde40eb5-dde8-474e-92db-d73d31ddc4fa None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:53:05.024584 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-bde40eb5-dde8-474e-92db-d73d31ddc4fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:53:05.178720 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 535/1079] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:53:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 11:53:05.471674 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28ea5f4-4a3a-4588-8cf9-95d3f7e71a2a None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:53:05.473619 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-a28ea5f4-4a3a-4588-8cf9-95d3f7e71a2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:53:05.615695 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:53:05.730423 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28ea5f4-4a3a-4588-8cf9-95d3f7e71a2a None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:53:05.914128 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9faf9f-3299-461c-a16c-6343f4b8d071 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bad5e1cb306b46818096940acc3810b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:53:05.914961 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9faf9f-3299-461c-a16c-6343f4b8d071 service nova] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:53:05.915075 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9faf9f-3299-461c-a16c-6343f4b8d071 service nova] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:53:05.915384 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9faf9f-3299-461c-a16c-6343f4b8d071 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:53:05.917135 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-af9faf9f-3299-461c-a16c-6343f4b8d071 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:53:05.970328 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:53:06.039149 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9faf9f-3299-461c-a16c-6343f4b8d071 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:53:06.040840 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9faf9f-3299-461c-a16c-6343f4b8d071 service nova] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:53:06.061303 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 545/1080] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 11:53:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 590 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 11:53:08.315406 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4499dbd0-aa3d-4e17-bc31-f927f2504cf0 None None] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:53:08.316316 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4499dbd0-aa3d-4e17-bc31-f927f2504cf0 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:53:08.316316 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4499dbd0-aa3d-4e17-bc31-f927f2504cf0 None None] PATH_INFO: `/` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:53:08.325600 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 536/1081] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:53:08 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:53:08.327216 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453f29a1-cdfc-4b22-ba5a-adcbe086b5ff None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:53:08.327930 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453f29a1-cdfc-4b22-ba5a-adcbe086b5ff None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:53:08.327930 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453f29a1-cdfc-4b22-ba5a-adcbe086b5ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:53:08.415941 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-453f29a1-cdfc-4b22-ba5a-adcbe086b5ff None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:53:08.417303 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-453f29a1-cdfc-4b22-ba5a-adcbe086b5ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:53:08.551143 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 546/1082] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:53:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 11:53:08.847587 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4499dbd0-aa3d-4e17-bc31-f927f2504cf0 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:53:08.849140 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-4499dbd0-aa3d-4e17-bc31-f927f2504cf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:53:09.019412 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:53:09.096069 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4499dbd0-aa3d-4e17-bc31-f927f2504cf0 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:53:09.252813 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc11ac5f-5120-40b3-9a9a-ae6605c56ca9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bad5e1cb306b46818096940acc3810b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:53:09.253423 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc11ac5f-5120-40b3-9a9a-ae6605c56ca9 service nova] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:53:09.253707 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc11ac5f-5120-40b3-9a9a-ae6605c56ca9 service nova] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:53:09.253984 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc11ac5f-5120-40b3-9a9a-ae6605c56ca9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:53:09.255701 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-fc11ac5f-5120-40b3-9a9a-ae6605c56ca9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:53:09.355677 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:53:09.380529 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc11ac5f-5120-40b3-9a9a-ae6605c56ca9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:53:09.382156 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc11ac5f-5120-40b3-9a9a-ae6605c56ca9 service nova] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:53:09.406780 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 537/1083] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 11:53:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 560 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 11:53:11.866940 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9551ece-f844-426d-86a3-6851c4886355 None None] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:53:11.867734 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9551ece-f844-426d-86a3-6851c4886355 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:53:11.868008 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9551ece-f844-426d-86a3-6851c4886355 None None] PATH_INFO: `/` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:53:11.870530 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 547/1084] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:53:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:53:11.880288 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3008864-bb43-43c5-89c7-9e8bc9208317 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:53:11.881053 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3008864-bb43-43c5-89c7-9e8bc9208317 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:53:11.881053 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3008864-bb43-43c5-89c7-9e8bc9208317 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:53:11.989295 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-d3008864-bb43-43c5-89c7-9e8bc9208317 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:53:11.992209 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-d3008864-bb43-43c5-89c7-9e8bc9208317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:53:12.154799 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 538/1085] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:53:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 11:53:12.482752 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9551ece-f844-426d-86a3-6851c4886355 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:53:12.484344 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-d9551ece-f844-426d-86a3-6851c4886355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:53:12.497886 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:53:12.731804 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9551ece-f844-426d-86a3-6851c4886355 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:53:12.835164 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:53:12.881908 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bce26b-e840-4746-9d03-6d3d1bff7d3f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bad5e1cb306b46818096940acc3810b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:53:12.883028 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bce26b-e840-4746-9d03-6d3d1bff7d3f service nova] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:53:12.883028 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bce26b-e840-4746-9d03-6d3d1bff7d3f service nova] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:53:12.883404 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bce26b-e840-4746-9d03-6d3d1bff7d3f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:53:12.884856 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-c4bce26b-e840-4746-9d03-6d3d1bff7d3f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:53:13.050427 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bce26b-e840-4746-9d03-6d3d1bff7d3f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:53:13.053717 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bce26b-e840-4746-9d03-6d3d1bff7d3f service nova] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:53:13.086738 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 548/1086] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 11:53:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 606 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 11:53:22.153081 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=79 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:53:23.082500 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=58 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:53:26.689279 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29a4d1f-6372-4a63-b838-5a441a1fc002 None None] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:53:26.689863 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29a4d1f-6372-4a63-b838-5a441a1fc002 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:53:26.689863 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29a4d1f-6372-4a63-b838-5a441a1fc002 None None] PATH_INFO: `/` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:53:26.691235 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 539/1087] 10.222.0.59 () {54 vars in 918 bytes} [Tue Oct 11 11:53:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:53:26.696803 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e2d0b7-36c4-4f3a-b2ed-47189ceb44c5 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:53:26.697291 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e2d0b7-36c4-4f3a-b2ed-47189ceb44c5 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:53:26.697398 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e2d0b7-36c4-4f3a-b2ed-47189ceb44c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:53:26.782387 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-01e2d0b7-36c4-4f3a-b2ed-47189ceb44c5 None None] MFA Rules not processed for user `f272cdd1bcc04b4e8261ed533cfd4fd7`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:53:26.783640 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-01e2d0b7-36c4-4f3a-b2ed-47189ceb44c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:53:26.909703 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 549/1088] 10.222.0.59 () {58 vars in 1015 bytes} [Tue Oct 11 11:53:26 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 11:53:26.924663 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36951d5-ed58-46d5-bf51-93e9a3684077 None None] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:53:26.924973 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36951d5-ed58-46d5-bf51-93e9a3684077 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:53:26.925242 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36951d5-ed58-46d5-bf51-93e9a3684077 None None] PATH_INFO: `/` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:53:26.926605 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 540/1089] 10.222.0.59 () {54 vars in 918 bytes} [Tue Oct 11 11:53:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:53:26.932223 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fb6ccb-78f2-4238-b19f-b0ca7eb613db None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:53:26.932494 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fb6ccb-78f2-4238-b19f-b0ca7eb613db None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:53:26.932790 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fb6ccb-78f2-4238-b19f-b0ca7eb613db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:53:27.013013 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:53:27.015760 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-59fb6ccb-78f2-4238-b19f-b0ca7eb613db None None] MFA Rules not processed for user `bad5e1cb306b46818096940acc3810b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:53:27.017456 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-59fb6ccb-78f2-4238-b19f-b0ca7eb613db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:53:27.146814 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 550/1090] 10.222.0.59 () {58 vars in 1015 bytes} [Tue Oct 11 11:53:26 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 11:53:29.711099 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d57728-f37e-481f-9f5e-b51014e508fc None None] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:53:29.711668 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d57728-f37e-481f-9f5e-b51014e508fc None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:53:29.711668 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d57728-f37e-481f-9f5e-b51014e508fc None None] PATH_INFO: `/` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:53:29.712994 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 541/1091] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 11 11:53:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:53:29.718782 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f47d23-22ae-4263-9ff7-dd78e0d1915b None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:53:29.719277 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f47d23-22ae-4263-9ff7-dd78e0d1915b None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:53:29.719277 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f47d23-22ae-4263-9ff7-dd78e0d1915b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:53:29.788522 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-09f47d23-22ae-4263-9ff7-dd78e0d1915b None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:53:29.789860 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-09f47d23-22ae-4263-9ff7-dd78e0d1915b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:53:29.852049 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:53:29.903506 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 551/1092] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 11 11:53:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 11:53:30.187289 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d57728-f37e-481f-9f5e-b51014e508fc None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:53:30.188923 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-70d57728-f37e-481f-9f5e-b51014e508fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:53:30.340024 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d57728-f37e-481f-9f5e-b51014e508fc None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:53:30.417369 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:53:30.457989 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951af605-6816-4c62-8c67-4184f33515cc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bad5e1cb306b46818096940acc3810b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:53:30.458686 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951af605-6816-4c62-8c67-4184f33515cc service nova] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:53:30.458953 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951af605-6816-4c62-8c67-4184f33515cc service nova] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:53:30.459230 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951af605-6816-4c62-8c67-4184f33515cc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:53:30.460925 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-951af605-6816-4c62-8c67-4184f33515cc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:53:30.572583 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951af605-6816-4c62-8c67-4184f33515cc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:53:30.574259 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951af605-6816-4c62-8c67-4184f33515cc service nova] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:53:30.596615 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 542/1093] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 11:53:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 11:53:31.761660 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc249bf-69af-462d-8c75-083e371326f0 None None] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:53:31.762223 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc249bf-69af-462d-8c75-083e371326f0 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:53:31.762223 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc249bf-69af-462d-8c75-083e371326f0 None None] PATH_INFO: `/` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:53:31.763674 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 552/1094] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 11 11:53:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:53:31.771446 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0291035-1039-4ab9-aa28-b132f23e049d None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:53:31.771977 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0291035-1039-4ab9-aa28-b132f23e049d None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:53:31.771977 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0291035-1039-4ab9-aa28-b132f23e049d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:53:31.861851 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-c0291035-1039-4ab9-aa28-b132f23e049d None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:53:31.863154 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-c0291035-1039-4ab9-aa28-b132f23e049d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:53:31.890701 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:53:31.970241 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 543/1095] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 11 11:53:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 11:53:32.117934 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc249bf-69af-462d-8c75-083e371326f0 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:53:32.119544 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-ebc249bf-69af-462d-8c75-083e371326f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:53:32.273435 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc249bf-69af-462d-8c75-083e371326f0 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:53:32.317932 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:53:32.397574 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518c1057-0b0a-4cda-bbbc-7c90771fbb49 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bad5e1cb306b46818096940acc3810b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:53:32.398102 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518c1057-0b0a-4cda-bbbc-7c90771fbb49 service nova] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:53:32.398385 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518c1057-0b0a-4cda-bbbc-7c90771fbb49 service nova] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:53:32.398647 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518c1057-0b0a-4cda-bbbc-7c90771fbb49 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:53:32.400306 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-518c1057-0b0a-4cda-bbbc-7c90771fbb49 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:53:32.493941 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518c1057-0b0a-4cda-bbbc-7c90771fbb49 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:53:32.495599 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518c1057-0b0a-4cda-bbbc-7c90771fbb49 service nova] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:53:32.516212 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 553/1096] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 11:53:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 11:53:33.646229 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85ab3b1-9a67-4b73-9c3a-4994a1b310f7 None None] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:53:33.646874 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85ab3b1-9a67-4b73-9c3a-4994a1b310f7 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:53:33.646874 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85ab3b1-9a67-4b73-9c3a-4994a1b310f7 None None] PATH_INFO: `/` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:53:33.648143 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 544/1097] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 11 11:53:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:53:33.655421 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3815d218-a2d2-4e7f-b938-b52585ca8acd None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:53:33.655876 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3815d218-a2d2-4e7f-b938-b52585ca8acd None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:53:33.656000 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3815d218-a2d2-4e7f-b938-b52585ca8acd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:53:33.718510 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:53:33.742879 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-3815d218-a2d2-4e7f-b938-b52585ca8acd None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:53:33.744111 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-3815d218-a2d2-4e7f-b938-b52585ca8acd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:53:33.850280 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 554/1098] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 11 11:53:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 11:53:34.004362 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85ab3b1-9a67-4b73-9c3a-4994a1b310f7 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:53:34.005897 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-f85ab3b1-9a67-4b73-9c3a-4994a1b310f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:53:34.158093 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85ab3b1-9a67-4b73-9c3a-4994a1b310f7 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:53:34.171046 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:53:34.288176 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a134c62a-e33e-4e2a-8cfd-483142cac11f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bad5e1cb306b46818096940acc3810b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:53:34.288926 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a134c62a-e33e-4e2a-8cfd-483142cac11f service nova] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:53:34.289321 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a134c62a-e33e-4e2a-8cfd-483142cac11f service nova] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:53:34.289712 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a134c62a-e33e-4e2a-8cfd-483142cac11f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:53:34.292244 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-a134c62a-e33e-4e2a-8cfd-483142cac11f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:53:34.393246 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a134c62a-e33e-4e2a-8cfd-483142cac11f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:53:34.394991 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a134c62a-e33e-4e2a-8cfd-483142cac11f service nova] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:53:34.417238 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 545/1099] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 11:53:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 11:53:35.532326 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4d9f68-08d7-40a7-9272-eacb74be8899 None None] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:53:35.532954 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4d9f68-08d7-40a7-9272-eacb74be8899 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:53:35.532954 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4d9f68-08d7-40a7-9272-eacb74be8899 None None] PATH_INFO: `/` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:53:35.534400 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 555/1100] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 11 11:53:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:53:35.541622 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb1c05d-a24a-4df1-a833-51b603223c2e None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:53:35.542200 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb1c05d-a24a-4df1-a833-51b603223c2e None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:53:35.542200 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb1c05d-a24a-4df1-a833-51b603223c2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:53:35.563319 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:53:35.624403 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-1cb1c05d-a24a-4df1-a833-51b603223c2e None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:53:35.625635 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-1cb1c05d-a24a-4df1-a833-51b603223c2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:53:35.729516 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 546/1101] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 11 11:53:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 11:53:35.877933 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4d9f68-08d7-40a7-9272-eacb74be8899 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:53:35.879814 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-5d4d9f68-08d7-40a7-9272-eacb74be8899 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:53:36.002897 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:53:36.041302 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4d9f68-08d7-40a7-9272-eacb74be8899 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:53:36.166891 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2fb718-7a5e-44ec-85d6-577fed69816d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bad5e1cb306b46818096940acc3810b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:53:36.167408 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2fb718-7a5e-44ec-85d6-577fed69816d service nova] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:53:36.167688 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2fb718-7a5e-44ec-85d6-577fed69816d service nova] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:53:36.167956 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2fb718-7a5e-44ec-85d6-577fed69816d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:53:36.169570 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-8b2fb718-7a5e-44ec-85d6-577fed69816d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:53:36.257406 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:53:36.275526 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2fb718-7a5e-44ec-85d6-577fed69816d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:53:36.277711 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2fb718-7a5e-44ec-85d6-577fed69816d service nova] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:53:36.298086 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 556/1102] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 11:53:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 11:53:37.425978 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8921b264-2507-4adc-a781-ac635312f678 None None] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:53:37.426655 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8921b264-2507-4adc-a781-ac635312f678 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:53:37.426655 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8921b264-2507-4adc-a781-ac635312f678 None None] PATH_INFO: `/` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:53:37.427936 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 547/1103] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 11 11:53:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:53:37.434815 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fbca5e-5ddb-4784-be70-85c126ebdd4b None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:53:37.435286 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fbca5e-5ddb-4784-be70-85c126ebdd4b None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:53:37.435396 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fbca5e-5ddb-4784-be70-85c126ebdd4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:53:37.521213 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-77fbca5e-5ddb-4784-be70-85c126ebdd4b None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:53:37.522588 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-77fbca5e-5ddb-4784-be70-85c126ebdd4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:53:37.628059 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 557/1104] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 11 11:53:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 11:53:37.777214 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8921b264-2507-4adc-a781-ac635312f678 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:53:37.779042 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-8921b264-2507-4adc-a781-ac635312f678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:53:37.874296 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:53:37.941166 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8921b264-2507-4adc-a781-ac635312f678 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:53:38.067951 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6d4c51-0b73-4ba3-808e-91c913bc2923 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bad5e1cb306b46818096940acc3810b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:53:38.068600 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6d4c51-0b73-4ba3-808e-91c913bc2923 service nova] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:53:38.068871 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6d4c51-0b73-4ba3-808e-91c913bc2923 service nova] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:53:38.069134 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6d4c51-0b73-4ba3-808e-91c913bc2923 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:53:38.070766 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-2e6d4c51-0b73-4ba3-808e-91c913bc2923 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:53:38.123089 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:53:38.168793 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6d4c51-0b73-4ba3-808e-91c913bc2923 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:53:38.170538 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6d4c51-0b73-4ba3-808e-91c913bc2923 service nova] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:53:38.190856 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 548/1105] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 11:53:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 11:53:39.266015 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1947ac-d2b4-46e2-a107-51f9fd0476dc None None] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:53:39.266621 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1947ac-d2b4-46e2-a107-51f9fd0476dc None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:53:39.266621 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1947ac-d2b4-46e2-a107-51f9fd0476dc None None] PATH_INFO: `/` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:53:39.267905 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 558/1106] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 11 11:53:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:53:39.274884 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be714bfe-f760-4be3-be50-12a1e85dc592 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:53:39.275435 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be714bfe-f760-4be3-be50-12a1e85dc592 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:53:39.275435 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be714bfe-f760-4be3-be50-12a1e85dc592 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:53:39.355103 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-be714bfe-f760-4be3-be50-12a1e85dc592 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:53:39.356329 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-be714bfe-f760-4be3-be50-12a1e85dc592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:53:39.458852 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 549/1107] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 11 11:53:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 11:53:39.608634 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1947ac-d2b4-46e2-a107-51f9fd0476dc None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:53:39.610184 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-bd1947ac-d2b4-46e2-a107-51f9fd0476dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:53:39.663725 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:53:39.769589 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1947ac-d2b4-46e2-a107-51f9fd0476dc None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:53:39.889126 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c9fe45-1d31-4e81-86e3-244564deeb57 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bad5e1cb306b46818096940acc3810b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:53:39.889643 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c9fe45-1d31-4e81-86e3-244564deeb57 service nova] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:53:39.889918 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c9fe45-1d31-4e81-86e3-244564deeb57 service nova] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:53:39.890191 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c9fe45-1d31-4e81-86e3-244564deeb57 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:53:39.891831 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-f8c9fe45-1d31-4e81-86e3-244564deeb57 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:53:39.899605 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:53:39.988914 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c9fe45-1d31-4e81-86e3-244564deeb57 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:53:39.990540 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c9fe45-1d31-4e81-86e3-244564deeb57 service nova] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:53:40.009671 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 559/1108] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 11:53:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 11:53:41.087585 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4f3c29-311a-4977-a1d7-de34854ed6e5 None None] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:53:41.088180 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4f3c29-311a-4977-a1d7-de34854ed6e5 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:53:41.088180 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4f3c29-311a-4977-a1d7-de34854ed6e5 None None] PATH_INFO: `/` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:53:41.089553 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 550/1109] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 11 11:53:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:53:41.096253 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2c0df1-56aa-4dc4-8b2b-699fd58a1b1f None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:53:41.096809 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2c0df1-56aa-4dc4-8b2b-699fd58a1b1f None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:53:41.096809 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2c0df1-56aa-4dc4-8b2b-699fd58a1b1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:53:41.179529 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-dc2c0df1-56aa-4dc4-8b2b-699fd58a1b1f None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:53:41.180776 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-dc2c0df1-56aa-4dc4-8b2b-699fd58a1b1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:53:41.250469 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:53:41.281807 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 560/1110] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 11 11:53:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 11:53:41.429266 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4f3c29-311a-4977-a1d7-de34854ed6e5 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:53:41.430847 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-2f4f3c29-311a-4977-a1d7-de34854ed6e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:53:41.449192 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:53:41.587514 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4f3c29-311a-4977-a1d7-de34854ed6e5 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:53:41.682900 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:53:41.710680 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d51b0a-1224-46f5-91d4-81dad1215cbb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bad5e1cb306b46818096940acc3810b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:53:41.711296 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d51b0a-1224-46f5-91d4-81dad1215cbb service nova] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:53:41.711577 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d51b0a-1224-46f5-91d4-81dad1215cbb service nova] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:53:41.711841 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d51b0a-1224-46f5-91d4-81dad1215cbb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:53:41.713467 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-f3d51b0a-1224-46f5-91d4-81dad1215cbb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:53:41.818691 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d51b0a-1224-46f5-91d4-81dad1215cbb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:53:41.820399 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d51b0a-1224-46f5-91d4-81dad1215cbb service nova] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:53:41.841277 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 551/1111] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 11:53:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 11:53:42.905825 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d3089d-894c-440f-a091-ca3fa5f8fe55 None None] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:53:42.906427 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d3089d-894c-440f-a091-ca3fa5f8fe55 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:53:42.906427 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d3089d-894c-440f-a091-ca3fa5f8fe55 None None] PATH_INFO: `/` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:53:42.907738 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 561/1112] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 11 11:53:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:53:42.915088 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4633ed06-0384-4ce6-872b-c3193de3f552 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:53:42.915608 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4633ed06-0384-4ce6-872b-c3193de3f552 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:53:42.915608 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4633ed06-0384-4ce6-872b-c3193de3f552 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:53:43.001210 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-4633ed06-0384-4ce6-872b-c3193de3f552 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:53:43.002492 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-4633ed06-0384-4ce6-872b-c3193de3f552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:53:43.047872 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:53:43.105679 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 552/1113] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 11 11:53:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 11:53:43.255704 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d3089d-894c-440f-a091-ca3fa5f8fe55 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:53:43.257236 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-e2d3089d-894c-440f-a091-ca3fa5f8fe55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:53:43.415738 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d3089d-894c-440f-a091-ca3fa5f8fe55 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:53:43.478530 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:53:43.549920 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0236f934-df0e-40d0-916c-2236b330d6f2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bad5e1cb306b46818096940acc3810b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:53:43.550595 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0236f934-df0e-40d0-916c-2236b330d6f2 service nova] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:53:43.550890 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0236f934-df0e-40d0-916c-2236b330d6f2 service nova] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:53:43.551167 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0236f934-df0e-40d0-916c-2236b330d6f2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:53:43.552898 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-0236f934-df0e-40d0-916c-2236b330d6f2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:53:43.699181 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0236f934-df0e-40d0-916c-2236b330d6f2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:53:43.700705 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0236f934-df0e-40d0-916c-2236b330d6f2 service nova] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:53:43.720447 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 562/1114] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 11:53:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 11 11:53:44.794927 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58aee1d-64d3-41c8-ae86-00006d1c9e0a None None] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:53:44.795546 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58aee1d-64d3-41c8-ae86-00006d1c9e0a None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:53:44.795546 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58aee1d-64d3-41c8-ae86-00006d1c9e0a None None] PATH_INFO: `/` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:53:44.796858 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 553/1115] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 11 11:53:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:53:44.803685 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab45cb41-33e1-4626-8cef-9498e27a4660 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:53:44.804180 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab45cb41-33e1-4626-8cef-9498e27a4660 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:53:44.804180 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab45cb41-33e1-4626-8cef-9498e27a4660 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:53:44.884458 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-ab45cb41-33e1-4626-8cef-9498e27a4660 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:53:44.885665 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-ab45cb41-33e1-4626-8cef-9498e27a4660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:53:44.887793 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:53:44.991007 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 563/1116] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 11 11:53:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 11:53:45.139689 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58aee1d-64d3-41c8-ae86-00006d1c9e0a None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:53:45.141108 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-c58aee1d-64d3-41c8-ae86-00006d1c9e0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:53:45.295813 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58aee1d-64d3-41c8-ae86-00006d1c9e0a None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:53:45.326224 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:53:45.417694 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4efa6d-1296-436c-9d69-2c9bfc9de78e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bad5e1cb306b46818096940acc3810b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:53:45.418203 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4efa6d-1296-436c-9d69-2c9bfc9de78e service nova] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:53:45.418480 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4efa6d-1296-436c-9d69-2c9bfc9de78e service nova] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:53:45.418788 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4efa6d-1296-436c-9d69-2c9bfc9de78e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:53:45.420412 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-ad4efa6d-1296-436c-9d69-2c9bfc9de78e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:53:45.516475 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4efa6d-1296-436c-9d69-2c9bfc9de78e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:53:45.518067 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4efa6d-1296-436c-9d69-2c9bfc9de78e service nova] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:53:45.536665 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 554/1117] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 11:53:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 11:53:47.627033 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc78a67c-ae88-47f3-b678-88c623a1b79e None None] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:53:47.627646 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc78a67c-ae88-47f3-b678-88c623a1b79e None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:53:47.627646 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc78a67c-ae88-47f3-b678-88c623a1b79e None None] PATH_INFO: `/` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:53:47.629078 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 564/1118] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 11 11:53:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:53:47.635246 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462d2c1a-8700-4191-a44e-fffd8ebb61df None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:53:47.635751 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462d2c1a-8700-4191-a44e-fffd8ebb61df None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:53:47.635846 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462d2c1a-8700-4191-a44e-fffd8ebb61df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:53:47.681215 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:53:47.710740 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-462d2c1a-8700-4191-a44e-fffd8ebb61df None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:53:47.712071 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-462d2c1a-8700-4191-a44e-fffd8ebb61df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:53:47.814500 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 555/1119] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 11 11:53:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 11:53:48.099745 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc78a67c-ae88-47f3-b678-88c623a1b79e None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:53:48.101325 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-bc78a67c-ae88-47f3-b678-88c623a1b79e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:53:48.212063 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc78a67c-ae88-47f3-b678-88c623a1b79e None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:53:48.245935 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:53:48.290966 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc51beb1-9ed9-448c-9f41-879ccdbd1f86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e563fe174db4cc19ae6bf8162cc3d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:53:48.291613 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc51beb1-9ed9-448c-9f41-879ccdbd1f86 service neutron] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:53:48.291893 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc51beb1-9ed9-448c-9f41-879ccdbd1f86 service neutron] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:53:48.292165 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc51beb1-9ed9-448c-9f41-879ccdbd1f86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:53:48.293905 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-fc51beb1-9ed9-448c-9f41-879ccdbd1f86 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:53:48.412896 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc51beb1-9ed9-448c-9f41-879ccdbd1f86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:53:48.414401 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc51beb1-9ed9-448c-9f41-879ccdbd1f86 service neutron] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:53:48.433186 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 565/1120] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 11:53:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 11:53:50.774524 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78964453-55f2-478a-a36d-eaf4e2b2fdf8 None None] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:53:50.775169 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78964453-55f2-478a-a36d-eaf4e2b2fdf8 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:53:50.775169 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78964453-55f2-478a-a36d-eaf4e2b2fdf8 None None] PATH_INFO: `/` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:53:50.776479 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 556/1121] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 11 11:53:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:53:50.782398 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ee8a56-46c9-4ded-8393-a4e5a48ca7c2 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:53:50.782894 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ee8a56-46c9-4ded-8393-a4e5a48ca7c2 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:53:50.783017 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ee8a56-46c9-4ded-8393-a4e5a48ca7c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:53:50.851749 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-e4ee8a56-46c9-4ded-8393-a4e5a48ca7c2 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:53:50.852994 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-e4ee8a56-46c9-4ded-8393-a4e5a48ca7c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:53:50.867215 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:53:50.961047 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 566/1122] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 11 11:53:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 11:53:51.243884 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78964453-55f2-478a-a36d-eaf4e2b2fdf8 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:53:51.245436 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-78964453-55f2-478a-a36d-eaf4e2b2fdf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:53:51.354192 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78964453-55f2-478a-a36d-eaf4e2b2fdf8 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:53:51.368262 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:53:51.432738 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16817223-6b2a-497f-8ab5-701cd9164e99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e563fe174db4cc19ae6bf8162cc3d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:53:51.433244 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16817223-6b2a-497f-8ab5-701cd9164e99 service neutron] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:53:51.433517 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16817223-6b2a-497f-8ab5-701cd9164e99 service neutron] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:53:51.433785 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16817223-6b2a-497f-8ab5-701cd9164e99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:53:51.435554 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-16817223-6b2a-497f-8ab5-701cd9164e99 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:53:51.555119 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16817223-6b2a-497f-8ab5-701cd9164e99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:53:51.556569 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16817223-6b2a-497f-8ab5-701cd9164e99 service neutron] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:53:51.575566 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 557/1123] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 11:53:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 11:53:54.073083 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8351eaa1-d587-4919-98d6-092811ea902d None None] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:53:54.073738 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8351eaa1-d587-4919-98d6-092811ea902d None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:53:54.073738 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8351eaa1-d587-4919-98d6-092811ea902d None None] PATH_INFO: `/` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:53:54.075180 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 567/1124] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 11 11:53:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:53:54.081463 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058ee032-f795-4c9c-9b4e-a90f7bcf08bb None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:53:54.081925 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058ee032-f795-4c9c-9b4e-a90f7bcf08bb None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:53:54.082017 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058ee032-f795-4c9c-9b4e-a90f7bcf08bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:53:54.135077 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:53:54.156159 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-058ee032-f795-4c9c-9b4e-a90f7bcf08bb None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:53:54.157527 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-058ee032-f795-4c9c-9b4e-a90f7bcf08bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:53:54.262611 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 558/1125] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 11 11:53:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 11:53:54.543993 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8351eaa1-d587-4919-98d6-092811ea902d None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:53:54.545687 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-8351eaa1-d587-4919-98d6-092811ea902d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:53:54.665517 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8351eaa1-d587-4919-98d6-092811ea902d None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:53:54.708437 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:53:54.743619 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ceeeda-3461-471e-bc24-722720379fdc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e563fe174db4cc19ae6bf8162cc3d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:53:54.744201 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ceeeda-3461-471e-bc24-722720379fdc service neutron] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:53:54.744459 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ceeeda-3461-471e-bc24-722720379fdc service neutron] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:53:54.744742 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ceeeda-3461-471e-bc24-722720379fdc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:53:54.746346 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-03ceeeda-3461-471e-bc24-722720379fdc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:53:54.863611 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ceeeda-3461-471e-bc24-722720379fdc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:53:54.865122 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ceeeda-3461-471e-bc24-722720379fdc service neutron] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:53:54.886837 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 568/1126] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 11:53:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 11:53:57.193487 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff93d86-b93a-46e1-9664-b1ca5c2aed9d None None] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:53:57.193973 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff93d86-b93a-46e1-9664-b1ca5c2aed9d None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:53:57.194087 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff93d86-b93a-46e1-9664-b1ca5c2aed9d None None] PATH_INFO: `/` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:53:57.195416 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 559/1127] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 11 11:53:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:53:57.201384 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd8be2b-53e5-44b0-8b6e-b0e47a87af3b None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:53:57.201826 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd8be2b-53e5-44b0-8b6e-b0e47a87af3b None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:53:57.201926 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd8be2b-53e5-44b0-8b6e-b0e47a87af3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:53:57.273970 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-6bd8be2b-53e5-44b0-8b6e-b0e47a87af3b None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:53:57.275145 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-6bd8be2b-53e5-44b0-8b6e-b0e47a87af3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:53:57.297606 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:53:57.375424 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 569/1128] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 11 11:53:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 11:53:57.656082 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff93d86-b93a-46e1-9664-b1ca5c2aed9d None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:53:57.657634 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-cff93d86-b93a-46e1-9664-b1ca5c2aed9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:53:57.770262 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff93d86-b93a-46e1-9664-b1ca5c2aed9d None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:53:57.792903 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:53:57.849654 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021f4308-560f-494d-9939-d008e680ae0b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e563fe174db4cc19ae6bf8162cc3d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:53:57.850260 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021f4308-560f-494d-9939-d008e680ae0b service neutron] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:53:57.850510 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021f4308-560f-494d-9939-d008e680ae0b service neutron] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:53:57.850837 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021f4308-560f-494d-9939-d008e680ae0b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:53:57.852438 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-021f4308-560f-494d-9939-d008e680ae0b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:53:57.972584 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021f4308-560f-494d-9939-d008e680ae0b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:53:57.974029 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021f4308-560f-494d-9939-d008e680ae0b service neutron] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:53:57.994501 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 560/1129] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 11:53:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 11:54:00.225365 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81ae789-8cfc-4bba-b124-70a611bfa9fe None None] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:54:00.225912 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81ae789-8cfc-4bba-b124-70a611bfa9fe None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:54:00.225912 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81ae789-8cfc-4bba-b124-70a611bfa9fe None None] PATH_INFO: `/` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:54:00.227231 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 570/1130] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 11 11:54:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:54:00.232818 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1f7395-d5f8-407a-943e-1d187ec48767 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:54:00.233273 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1f7395-d5f8-407a-943e-1d187ec48767 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:54:00.233371 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1f7395-d5f8-407a-943e-1d187ec48767 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:54:00.299753 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:54:00.303172 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-0c1f7395-d5f8-407a-943e-1d187ec48767 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:54:00.305363 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-0c1f7395-d5f8-407a-943e-1d187ec48767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:54:00.408077 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 561/1131] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 11 11:54:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 11:54:00.694543 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81ae789-8cfc-4bba-b124-70a611bfa9fe None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:54:00.696158 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-c81ae789-8cfc-4bba-b124-70a611bfa9fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:54:00.804757 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81ae789-8cfc-4bba-b124-70a611bfa9fe None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:54:00.856220 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:54:00.882544 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f91a7b-b129-4308-b12d-2958fad9246c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e563fe174db4cc19ae6bf8162cc3d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:54:00.883213 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f91a7b-b129-4308-b12d-2958fad9246c service neutron] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:54:00.883479 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f91a7b-b129-4308-b12d-2958fad9246c service neutron] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:54:00.883754 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f91a7b-b129-4308-b12d-2958fad9246c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:54:00.885368 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-c4f91a7b-b129-4308-b12d-2958fad9246c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:54:01.004163 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f91a7b-b129-4308-b12d-2958fad9246c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:54:01.005618 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f91a7b-b129-4308-b12d-2958fad9246c service neutron] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:54:01.024683 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 571/1132] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 11:54:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 11:54:03.249422 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dd5f73-43ac-45ea-81bb-294f7ca7b5ed None None] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:54:03.250024 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dd5f73-43ac-45ea-81bb-294f7ca7b5ed None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:54:03.250024 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dd5f73-43ac-45ea-81bb-294f7ca7b5ed None None] PATH_INFO: `/` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:54:03.251422 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 562/1133] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 11 11:54:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:54:03.257555 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b5f5a7-a8bc-47e8-ab2f-fbfc6d4d7f09 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:54:03.257980 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b5f5a7-a8bc-47e8-ab2f-fbfc6d4d7f09 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:54:03.258093 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b5f5a7-a8bc-47e8-ab2f-fbfc6d4d7f09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:54:03.334797 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-c2b5f5a7-a8bc-47e8-ab2f-fbfc6d4d7f09 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:54:03.336035 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-c2b5f5a7-a8bc-47e8-ab2f-fbfc6d4d7f09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:54:03.368034 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:54:03.438601 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 572/1134] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 11 11:54:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 11:54:03.718533 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dd5f73-43ac-45ea-81bb-294f7ca7b5ed None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:54:03.720254 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-e8dd5f73-43ac-45ea-81bb-294f7ca7b5ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:54:03.832347 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dd5f73-43ac-45ea-81bb-294f7ca7b5ed None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:54:03.860971 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:54:03.909320 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d364f1df-33b6-42a4-a71c-6913c97026fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e563fe174db4cc19ae6bf8162cc3d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:54:03.909908 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d364f1df-33b6-42a4-a71c-6913c97026fb service neutron] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:54:03.910204 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d364f1df-33b6-42a4-a71c-6913c97026fb service neutron] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:54:03.910470 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d364f1df-33b6-42a4-a71c-6913c97026fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:54:03.912086 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-d364f1df-33b6-42a4-a71c-6913c97026fb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:54:04.028561 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d364f1df-33b6-42a4-a71c-6913c97026fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:54:04.030017 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d364f1df-33b6-42a4-a71c-6913c97026fb service neutron] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:54:04.049340 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 563/1135] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 11:54:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 11:54:06.363343 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb98ed70-c6e9-426d-ac84-34419edb98a3 None None] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:54:06.363959 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb98ed70-c6e9-426d-ac84-34419edb98a3 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:54:06.363959 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb98ed70-c6e9-426d-ac84-34419edb98a3 None None] PATH_INFO: `/` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:54:06.365218 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 573/1136] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 11 11:54:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 11 11:54:06.372352 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a02e78-c7b7-4143-be8d-ab9b3dc6bfed None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:54:06.372939 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a02e78-c7b7-4143-be8d-ab9b3dc6bfed None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:54:06.372939 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a02e78-c7b7-4143-be8d-ab9b3dc6bfed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:54:06.445206 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-e9a02e78-c7b7-4143-be8d-ab9b3dc6bfed None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:54:06.446442 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-e9a02e78-c7b7-4143-be8d-ab9b3dc6bfed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:54:06.455940 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:54:06.551254 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 564/1137] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 11 11:54:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 11:54:06.833049 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb98ed70-c6e9-426d-ac84-34419edb98a3 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:54:06.834613 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-bb98ed70-c6e9-426d-ac84-34419edb98a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:54:06.942839 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb98ed70-c6e9-426d-ac84-34419edb98a3 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:54:07.003003 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:54:07.021477 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8542b4a3-12bf-4eae-abd7-90d731db457c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e563fe174db4cc19ae6bf8162cc3d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:54:07.022143 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8542b4a3-12bf-4eae-abd7-90d731db457c service neutron] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:54:07.022405 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8542b4a3-12bf-4eae-abd7-90d731db457c service neutron] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:54:07.022706 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8542b4a3-12bf-4eae-abd7-90d731db457c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:54:07.024324 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-8542b4a3-12bf-4eae-abd7-90d731db457c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:54:07.138624 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8542b4a3-12bf-4eae-abd7-90d731db457c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:54:07.140033 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8542b4a3-12bf-4eae-abd7-90d731db457c service neutron] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:54:07.159822 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 574/1138] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 11:54:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 11:54:09.366325 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e205b6d-26a2-4830-842b-27f96f933c6e None None] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:54:09.367008 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e205b6d-26a2-4830-842b-27f96f933c6e None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:54:09.367008 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e205b6d-26a2-4830-842b-27f96f933c6e None None] PATH_INFO: `/` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:54:09.368258 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 565/1139] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 11 11:54:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:54:09.374076 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101d1184-137c-4768-a3f7-8755abe0c3dd None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:54:09.374546 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101d1184-137c-4768-a3f7-8755abe0c3dd None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:54:09.374685 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101d1184-137c-4768-a3f7-8755abe0c3dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:54:09.448927 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-101d1184-137c-4768-a3f7-8755abe0c3dd None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:54:09.450199 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-101d1184-137c-4768-a3f7-8755abe0c3dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:54:09.490948 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:54:09.551144 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 575/1140] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 11 11:54:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 11:54:09.833074 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e205b6d-26a2-4830-842b-27f96f933c6e None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:54:09.834761 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-5e205b6d-26a2-4830-842b-27f96f933c6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:54:09.945013 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e205b6d-26a2-4830-842b-27f96f933c6e None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:54:09.986764 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:54:10.026947 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a083042a-ba5e-4179-b01a-e32eb4fc1f67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e563fe174db4cc19ae6bf8162cc3d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:54:10.027557 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a083042a-ba5e-4179-b01a-e32eb4fc1f67 service neutron] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:54:10.027840 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a083042a-ba5e-4179-b01a-e32eb4fc1f67 service neutron] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:54:10.028110 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a083042a-ba5e-4179-b01a-e32eb4fc1f67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:54:10.030041 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-a083042a-ba5e-4179-b01a-e32eb4fc1f67 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:54:10.145569 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a083042a-ba5e-4179-b01a-e32eb4fc1f67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:54:10.146987 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a083042a-ba5e-4179-b01a-e32eb4fc1f67 service neutron] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:54:10.166059 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 566/1141] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 11:54:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 11:54:12.355105 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52c965d-8871-475d-80be-a3d7dd7bb6b6 None None] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:54:12.355713 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52c965d-8871-475d-80be-a3d7dd7bb6b6 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:54:12.355713 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52c965d-8871-475d-80be-a3d7dd7bb6b6 None None] PATH_INFO: `/` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:54:12.357013 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 576/1142] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 11 11:54:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:54:12.362738 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff718ee2-80e6-4fda-97f9-b2462f55a02e None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:54:12.363178 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff718ee2-80e6-4fda-97f9-b2462f55a02e None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:54:12.363290 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff718ee2-80e6-4fda-97f9-b2462f55a02e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:54:12.436107 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-ff718ee2-80e6-4fda-97f9-b2462f55a02e None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:54:12.437329 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-ff718ee2-80e6-4fda-97f9-b2462f55a02e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:54:12.457748 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:54:12.541288 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 567/1143] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 11 11:54:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 11:54:12.825982 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52c965d-8871-475d-80be-a3d7dd7bb6b6 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:54:12.827583 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-c52c965d-8871-475d-80be-a3d7dd7bb6b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:54:12.939236 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52c965d-8871-475d-80be-a3d7dd7bb6b6 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:54:13.011870 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:54:13.019588 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa97d11-b0e0-4e99-9782-c041f4d1952f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e563fe174db4cc19ae6bf8162cc3d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:54:13.020210 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa97d11-b0e0-4e99-9782-c041f4d1952f service neutron] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:54:13.020488 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa97d11-b0e0-4e99-9782-c041f4d1952f service neutron] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:54:13.020772 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa97d11-b0e0-4e99-9782-c041f4d1952f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:54:13.022427 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-3aa97d11-b0e0-4e99-9782-c041f4d1952f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:54:13.137989 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa97d11-b0e0-4e99-9782-c041f4d1952f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:54:13.139507 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa97d11-b0e0-4e99-9782-c041f4d1952f service neutron] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:54:13.158066 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 577/1144] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 11:54:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 11:54:15.342392 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde847e2-750a-43ee-8b86-fef3384d62ec None None] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:54:15.343045 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde847e2-750a-43ee-8b86-fef3384d62ec None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:54:15.343045 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde847e2-750a-43ee-8b86-fef3384d62ec None None] PATH_INFO: `/` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:54:15.344330 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 568/1145] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 11 11:54:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:54:15.350025 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b8e96e-bc9a-4867-8d93-d63931cc4f22 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:54:15.350478 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b8e96e-bc9a-4867-8d93-d63931cc4f22 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:54:15.350627 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b8e96e-bc9a-4867-8d93-d63931cc4f22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:54:15.422602 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-c3b8e96e-bc9a-4867-8d93-d63931cc4f22 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:54:15.423833 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-c3b8e96e-bc9a-4867-8d93-d63931cc4f22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:54:15.473459 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:54:15.521943 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 578/1146] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 11 11:54:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 11:54:15.798354 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde847e2-750a-43ee-8b86-fef3384d62ec None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:54:15.799988 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-bde847e2-750a-43ee-8b86-fef3384d62ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:54:15.906517 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde847e2-750a-43ee-8b86-fef3384d62ec None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:54:15.955893 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:54:15.985442 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e995c83e-20fe-4e53-9859-2140088e9f3a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e563fe174db4cc19ae6bf8162cc3d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:54:15.986053 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e995c83e-20fe-4e53-9859-2140088e9f3a service neutron] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:54:15.986336 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e995c83e-20fe-4e53-9859-2140088e9f3a service neutron] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:54:15.986601 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e995c83e-20fe-4e53-9859-2140088e9f3a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:54:15.988251 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-e995c83e-20fe-4e53-9859-2140088e9f3a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:54:16.105456 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e995c83e-20fe-4e53-9859-2140088e9f3a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:54:16.106815 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e995c83e-20fe-4e53-9859-2140088e9f3a service neutron] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:54:16.125637 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 569/1147] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 11:54:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 11:54:25.520315 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=22 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:54:26.123200 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=66 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:54:36.669882 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d812854-1e2d-4a92-989f-2ce5033e72b5 None None] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:54:36.670629 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d812854-1e2d-4a92-989f-2ce5033e72b5 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:54:36.670629 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d812854-1e2d-4a92-989f-2ce5033e72b5 None None] PATH_INFO: `/` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:54:36.671803 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 579/1148] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:54:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:54:36.677597 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0fd83f-90f4-4633-a4a0-7c34f306f2ad None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:54:36.678039 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0fd83f-90f4-4633-a4a0-7c34f306f2ad None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:54:36.678155 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0fd83f-90f4-4633-a4a0-7c34f306f2ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:54:36.747182 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-9b0fd83f-90f4-4633-a4a0-7c34f306f2ad None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:54:36.748486 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-9b0fd83f-90f4-4633-a4a0-7c34f306f2ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:54:36.859927 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 570/1149] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:54:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 11:54:37.154087 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4327eb77-c4c2-4439-a59b-d8e9128f5360 None None] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:54:37.154373 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4327eb77-c4c2-4439-a59b-d8e9128f5360 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:54:37.154692 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4327eb77-c4c2-4439-a59b-d8e9128f5360 None None] PATH_INFO: `/` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:54:37.156067 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 580/1150] 10.222.0.59 () {54 vars in 962 bytes} [Tue Oct 11 11:54:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:54:37.161213 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8b93f2-824f-4c5b-a361-f922d8790e82 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:54:37.161501 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8b93f2-824f-4c5b-a361-f922d8790e82 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:54:37.161775 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8b93f2-824f-4c5b-a361-f922d8790e82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:54:37.241693 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-de8b93f2-824f-4c5b-a361-f922d8790e82 None None] MFA Rules not processed for user `91a00658b79a42bd91a22a2508b5de34`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:54:37.242887 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-de8b93f2-824f-4c5b-a361-f922d8790e82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:54:37.263298 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:54:37.320997 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 571/1151] 10.222.0.59 () {58 vars in 1059 bytes} [Tue Oct 11 11:54:37 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 11:54:37.329348 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4327eb77-c4c2-4439-a59b-d8e9128f5360 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:54:37.331023 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-4327eb77-c4c2-4439-a59b-d8e9128f5360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:54:37.444257 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4327eb77-c4c2-4439-a59b-d8e9128f5360 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:54:37.523564 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d2db9d-3571-43c3-b0c7-6e8dfcdce36b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91a00658b79a42bd91a22a2508b5de34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:54:37.524129 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d2db9d-3571-43c3-b0c7-6e8dfcdce36b service glance] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:54:37.524404 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d2db9d-3571-43c3-b0c7-6e8dfcdce36b service glance] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:54:37.524671 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d2db9d-3571-43c3-b0c7-6e8dfcdce36b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:54:37.526270 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-f3d2db9d-3571-43c3-b0c7-6e8dfcdce36b service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:54:37.581206 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:54:37.637281 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d2db9d-3571-43c3-b0c7-6e8dfcdce36b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:54:37.638746 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d2db9d-3571-43c3-b0c7-6e8dfcdce36b service glance] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:54:37.658782 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 581/1152] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 11:54:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 11:54:39.906407 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4323d9fc-ccee-4fb7-bc95-a9d5eca1b3ef None None] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:54:39.907093 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4323d9fc-ccee-4fb7-bc95-a9d5eca1b3ef None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:54:39.907093 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4323d9fc-ccee-4fb7-bc95-a9d5eca1b3ef None None] PATH_INFO: `/` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:54:39.908335 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 572/1153] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:54:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:54:39.914427 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aece86-f26e-4be5-bf9e-e1f5dc88b9c4 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:54:39.914939 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aece86-f26e-4be5-bf9e-e1f5dc88b9c4 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:54:39.915048 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aece86-f26e-4be5-bf9e-e1f5dc88b9c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:54:39.986721 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-f0aece86-f26e-4be5-bf9e-e1f5dc88b9c4 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:54:39.988018 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-f0aece86-f26e-4be5-bf9e-e1f5dc88b9c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:54:40.088469 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 582/1154] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:54:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 11:54:40.359317 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4323d9fc-ccee-4fb7-bc95-a9d5eca1b3ef None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:54:40.360876 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-4323d9fc-ccee-4fb7-bc95-a9d5eca1b3ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:54:40.495903 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4323d9fc-ccee-4fb7-bc95-a9d5eca1b3ef None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:54:40.548446 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:54:40.597361 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99add225-6e95-4d5d-a8e8-0cfe320b7cdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:54:40.597970 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99add225-6e95-4d5d-a8e8-0cfe320b7cdd admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:54:40.598250 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99add225-6e95-4d5d-a8e8-0cfe320b7cdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:54:40.598521 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99add225-6e95-4d5d-a8e8-0cfe320b7cdd admin admin] PATH_INFO: `/v3/projects` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:54:40.599179 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99add225-6e95-4d5d-a8e8-0cfe320b7cdd admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:54:40.600283 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: WARNING py.warnings [None req-99add225-6e95-4d5d-a8e8-0cfe320b7cdd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 11 11:54:40.600283 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: warnings.warn(msg) Oct 11 11:54:40.600283 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: Oct 11 11:54:40.601420 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99add225-6e95-4d5d-a8e8-0cfe320b7cdd admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:54:40.621879 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 573/1155] 10.222.0.59 () {56 vars in 1108 bytes} [Tue Oct 11 11:54:40 2022] GET /identity/v3/projects => generated 2553 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 11:54:42.817856 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd442fbf-a582-4d38-965f-5c1795bc23c3 None None] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:54:42.818479 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd442fbf-a582-4d38-965f-5c1795bc23c3 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:54:42.818479 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd442fbf-a582-4d38-965f-5c1795bc23c3 None None] PATH_INFO: `/` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:54:42.819791 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 583/1156] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:54:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:54:42.825539 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169b3791-b061-4f0a-b9a8-d470e9f5018b None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:54:42.826035 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169b3791-b061-4f0a-b9a8-d470e9f5018b None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:54:42.826159 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169b3791-b061-4f0a-b9a8-d470e9f5018b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:54:42.896848 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-169b3791-b061-4f0a-b9a8-d470e9f5018b None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:54:42.898069 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-169b3791-b061-4f0a-b9a8-d470e9f5018b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:54:43.001028 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 574/1157] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:54:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 11:54:43.294835 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd442fbf-a582-4d38-965f-5c1795bc23c3 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:54:43.296437 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-bd442fbf-a582-4d38-965f-5c1795bc23c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:54:43.306808 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:54:43.452872 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd442fbf-a582-4d38-965f-5c1795bc23c3 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:54:43.537508 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:54:43.571854 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bdba1e-3fb9-4a8a-88bc-0bc52e0f9747 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bad5e1cb306b46818096940acc3810b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:54:43.572438 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bdba1e-3fb9-4a8a-88bc-0bc52e0f9747 service nova] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:54:43.572812 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bdba1e-3fb9-4a8a-88bc-0bc52e0f9747 service nova] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:54:43.573077 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bdba1e-3fb9-4a8a-88bc-0bc52e0f9747 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:54:43.574673 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-03bdba1e-3fb9-4a8a-88bc-0bc52e0f9747 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:54:43.675583 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bdba1e-3fb9-4a8a-88bc-0bc52e0f9747 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:54:43.677242 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bdba1e-3fb9-4a8a-88bc-0bc52e0f9747 service nova] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:54:43.696645 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 584/1158] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 11:54:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 11 11:54:46.119177 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068eec43-8327-4404-9c59-d226fdb4b567 None None] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:54:46.119766 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068eec43-8327-4404-9c59-d226fdb4b567 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:54:46.119766 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068eec43-8327-4404-9c59-d226fdb4b567 None None] PATH_INFO: `/` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:54:46.121241 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 575/1159] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:54:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:54:46.126789 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf654c9a-cdf8-433a-92c2-fc6bd14ecb79 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:54:46.127234 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf654c9a-cdf8-433a-92c2-fc6bd14ecb79 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:54:46.127344 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf654c9a-cdf8-433a-92c2-fc6bd14ecb79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:54:46.198395 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-bf654c9a-cdf8-433a-92c2-fc6bd14ecb79 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:54:46.199658 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-bf654c9a-cdf8-433a-92c2-fc6bd14ecb79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:54:46.250929 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:54:46.306571 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 585/1160] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:54:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 11:54:46.595421 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068eec43-8327-4404-9c59-d226fdb4b567 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:54:46.597192 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-068eec43-8327-4404-9c59-d226fdb4b567 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:54:46.623603 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:54:46.710151 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068eec43-8327-4404-9c59-d226fdb4b567 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:54:46.793493 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ed7d78-0cc6-4b8f-8def-cae4f471b7da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e563fe174db4cc19ae6bf8162cc3d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:54:46.794225 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ed7d78-0cc6-4b8f-8def-cae4f471b7da service neutron] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:54:46.794504 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ed7d78-0cc6-4b8f-8def-cae4f471b7da service neutron] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:54:46.794804 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ed7d78-0cc6-4b8f-8def-cae4f471b7da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:54:46.796431 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-a6ed7d78-0cc6-4b8f-8def-cae4f471b7da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:54:46.879853 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:54:46.915894 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ed7d78-0cc6-4b8f-8def-cae4f471b7da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:54:46.917437 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ed7d78-0cc6-4b8f-8def-cae4f471b7da service neutron] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:54:46.937571 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 576/1161] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 11:54:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 11:54:49.149781 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d420ec-0367-4800-8573-cf6dde2b3b5d None None] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:54:49.150364 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d420ec-0367-4800-8573-cf6dde2b3b5d None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:54:49.150364 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d420ec-0367-4800-8573-cf6dde2b3b5d None None] PATH_INFO: `/` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:54:49.151807 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 586/1162] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:54:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:54:49.158205 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bdabda-603b-4b07-92f6-557d1af8d35c None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:54:49.158726 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bdabda-603b-4b07-92f6-557d1af8d35c None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:54:49.158840 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bdabda-603b-4b07-92f6-557d1af8d35c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:54:49.228647 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-13bdabda-603b-4b07-92f6-557d1af8d35c None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:54:49.229773 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-13bdabda-603b-4b07-92f6-557d1af8d35c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:54:49.333847 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 577/1163] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:54:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 11:54:49.617390 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d420ec-0367-4800-8573-cf6dde2b3b5d None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:54:49.618999 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-50d420ec-0367-4800-8573-cf6dde2b3b5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:54:49.729744 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d420ec-0367-4800-8573-cf6dde2b3b5d None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:54:49.809621 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:54:49.811208 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b4cd2e-e769-4a96-92f3-24c17350cb9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e563fe174db4cc19ae6bf8162cc3d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:54:49.811940 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b4cd2e-e769-4a96-92f3-24c17350cb9d service neutron] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:54:49.812248 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b4cd2e-e769-4a96-92f3-24c17350cb9d service neutron] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:54:49.812535 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b4cd2e-e769-4a96-92f3-24c17350cb9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:54:49.814787 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-04b4cd2e-e769-4a96-92f3-24c17350cb9d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:54:49.942646 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b4cd2e-e769-4a96-92f3-24c17350cb9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:54:49.944066 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b4cd2e-e769-4a96-92f3-24c17350cb9d service neutron] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:54:49.967145 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 587/1164] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 11:54:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 11:54:52.281906 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c551cfa3-1945-4406-aacb-423e31039a72 None None] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:54:52.282475 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c551cfa3-1945-4406-aacb-423e31039a72 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:54:52.282475 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c551cfa3-1945-4406-aacb-423e31039a72 None None] PATH_INFO: `/` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:54:52.283741 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 578/1165] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:54:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:54:52.289268 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990db06e-7102-4eed-8192-ab76ef7b043a None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:54:52.289718 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990db06e-7102-4eed-8192-ab76ef7b043a None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:54:52.289831 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990db06e-7102-4eed-8192-ab76ef7b043a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:54:52.358155 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-990db06e-7102-4eed-8192-ab76ef7b043a None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:54:52.359424 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-990db06e-7102-4eed-8192-ab76ef7b043a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:54:52.416854 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:54:52.464133 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 588/1166] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:54:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 11:54:52.749613 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c551cfa3-1945-4406-aacb-423e31039a72 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:54:52.751226 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-c551cfa3-1945-4406-aacb-423e31039a72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:54:52.789751 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:54:52.860459 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c551cfa3-1945-4406-aacb-423e31039a72 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:54:52.951384 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f691d308-f6b5-4d7e-8ea8-44a0bf689033 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e563fe174db4cc19ae6bf8162cc3d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:54:52.952054 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f691d308-f6b5-4d7e-8ea8-44a0bf689033 service neutron] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:54:52.952342 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f691d308-f6b5-4d7e-8ea8-44a0bf689033 service neutron] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:54:52.952611 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f691d308-f6b5-4d7e-8ea8-44a0bf689033 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:54:52.954314 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-f691d308-f6b5-4d7e-8ea8-44a0bf689033 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:54:53.045010 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:54:53.076592 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f691d308-f6b5-4d7e-8ea8-44a0bf689033 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:54:53.078032 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f691d308-f6b5-4d7e-8ea8-44a0bf689033 service neutron] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:54:53.097664 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 579/1167] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 11:54:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 11:54:55.547940 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0d003d-d979-4007-8927-3f570de6db83 None None] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:54:55.548529 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0d003d-d979-4007-8927-3f570de6db83 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:54:55.548529 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0d003d-d979-4007-8927-3f570de6db83 None None] PATH_INFO: `/` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:54:55.549901 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 589/1168] 10.222.0.59 () {54 vars in 927 bytes} [Tue Oct 11 11:54:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:54:55.555396 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2570fb1-f3d4-42ca-bb99-6e4bfbdaea33 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:54:55.555874 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2570fb1-f3d4-42ca-bb99-6e4bfbdaea33 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:54:55.555976 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2570fb1-f3d4-42ca-bb99-6e4bfbdaea33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:54:55.628705 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-f2570fb1-f3d4-42ca-bb99-6e4bfbdaea33 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:54:55.629952 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-f2570fb1-f3d4-42ca-bb99-6e4bfbdaea33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:54:55.733898 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 580/1169] 10.222.0.59 () {58 vars in 1024 bytes} [Tue Oct 11 11:54:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 11:54:56.021956 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0d003d-d979-4007-8927-3f570de6db83 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:54:56.023687 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-ac0d003d-d979-4007-8927-3f570de6db83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:54:56.138994 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0d003d-d979-4007-8927-3f570de6db83 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:54:56.220549 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b107b41-095c-470f-a4ed-63fd2270232b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e563fe174db4cc19ae6bf8162cc3d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:54:56.221117 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b107b41-095c-470f-a4ed-63fd2270232b service neutron] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:54:56.221464 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b107b41-095c-470f-a4ed-63fd2270232b service neutron] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:54:56.221751 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b107b41-095c-470f-a4ed-63fd2270232b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:54:56.223521 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-9b107b41-095c-470f-a4ed-63fd2270232b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:54:56.233147 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:54:56.345630 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b107b41-095c-470f-a4ed-63fd2270232b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:54:56.347409 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b107b41-095c-470f-a4ed-63fd2270232b service neutron] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:54:56.367183 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 590/1170] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 11:54:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 11:55:05.732213 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=82 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:55:06.364300 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=50 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:56:16.331751 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bd93e9-88d0-4fae-a787-d679c5f12ed5 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:56:16.332359 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bd93e9-88d0-4fae-a787-d679c5f12ed5 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:56:16.332359 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bd93e9-88d0-4fae-a787-d679c5f12ed5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:56:16.416861 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-05bd93e9-88d0-4fae-a787-d679c5f12ed5 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:56:16.418097 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-05bd93e9-88d0-4fae-a787-d679c5f12ed5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:56:16.529587 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 581/1171] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 11:56:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 11:56:16.539870 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49036b7-4a57-42ce-90f3-dd268dff600d None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:56:16.540427 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49036b7-4a57-42ce-90f3-dd268dff600d None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:56:16.540530 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49036b7-4a57-42ce-90f3-dd268dff600d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:56:16.639488 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-e49036b7-4a57-42ce-90f3-dd268dff600d None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:56:16.640843 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-e49036b7-4a57-42ce-90f3-dd268dff600d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:56:16.741449 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 591/1172] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 11:56:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 11:56:16.746548 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bd93e9-88d0-4fae-a787-d679c5f12ed5 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:56:16.747794 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-05bd93e9-88d0-4fae-a787-d679c5f12ed5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:56:16.759309 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bd93e9-88d0-4fae-a787-d679c5f12ed5 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:56:16.828843 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:56:16.864709 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab17494-7e31-4037-8c18-48ff849d73ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:56:16.865298 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab17494-7e31-4037-8c18-48ff849d73ad admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:56:16.865606 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab17494-7e31-4037-8c18-48ff849d73ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:56:16.865874 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab17494-7e31-4037-8c18-48ff849d73ad admin admin] PATH_INFO: `/v3/domains` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:56:16.866536 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab17494-7e31-4037-8c18-48ff849d73ad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:56:16.866873 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab17494-7e31-4037-8c18-48ff849d73ad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:56:16.867907 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: WARNING py.warnings [None req-2ab17494-7e31-4037-8c18-48ff849d73ad admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 11 11:56:16.867907 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: warnings.warn(msg) Oct 11 11:56:16.867907 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: Oct 11 11:56:16.868802 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab17494-7e31-4037-8c18-48ff849d73ad admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:56:16.881153 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 582/1173] 10.222.0.59 () {58 vars in 1153 bytes} [Tue Oct 11 11:56:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:56:16.891036 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49036b7-4a57-42ce-90f3-dd268dff600d None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:56:16.900957 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49036b7-4a57-42ce-90f3-dd268dff600d None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:56:16.972014 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:56:17.006415 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0105db-07d7-4208-8e2e-f6a76190e6b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:56:17.007048 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0105db-07d7-4208-8e2e-f6a76190e6b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:56:17.007324 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0105db-07d7-4208-8e2e-f6a76190e6b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:56:17.007597 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0105db-07d7-4208-8e2e-f6a76190e6b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:56:17.008224 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0105db-07d7-4208-8e2e-f6a76190e6b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:56:17.009245 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: WARNING py.warnings [None req-7e0105db-07d7-4208-8e2e-f6a76190e6b4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 11 11:56:17.009245 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: warnings.warn(msg) Oct 11 11:56:17.009245 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: Oct 11 11:56:17.010441 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0105db-07d7-4208-8e2e-f6a76190e6b4 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:56:17.075248 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 592/1174] 10.222.0.59 () {60 vars in 1153 bytes} [Tue Oct 11 11:56:16 2022] POST /identity/v3/projects => generated 377 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 11:56:17.079670 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab17494-7e31-4037-8c18-48ff849d73ad admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:56:17.081231 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-2ab17494-7e31-4037-8c18-48ff849d73ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:56:17.117524 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab17494-7e31-4037-8c18-48ff849d73ad admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:56:17.222641 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c332bbed-c1ac-4dee-baa1-f2f673805020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:56:17.223271 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c332bbed-c1ac-4dee-baa1-f2f673805020 admin admin] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:56:17.223553 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c332bbed-c1ac-4dee-baa1-f2f673805020 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:56:17.223820 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c332bbed-c1ac-4dee-baa1-f2f673805020 admin admin] PATH_INFO: `/v3/users` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:56:17.224458 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c332bbed-c1ac-4dee-baa1-f2f673805020 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:56:17.225469 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: WARNING py.warnings [None req-c332bbed-c1ac-4dee-baa1-f2f673805020 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 11 11:56:17.225469 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: warnings.warn(msg) Oct 11 11:56:17.225469 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: Oct 11 11:56:17.226713 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c332bbed-c1ac-4dee-baa1-f2f673805020 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:56:17.273894 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 583/1175] 10.222.0.59 () {60 vars in 1144 bytes} [Tue Oct 11 11:56:17 2022] POST /identity/v3/users => generated 289 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 11:56:17.278118 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0105db-07d7-4208-8e2e-f6a76190e6b4 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:56:17.279986 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-7e0105db-07d7-4208-8e2e-f6a76190e6b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:56:17.349477 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0105db-07d7-4208-8e2e-f6a76190e6b4 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:56:17.473492 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:56:17.493932 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bcd775-3480-4d2a-8cd5-28e551298c8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:56:17.495252 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bcd775-3480-4d2a-8cd5-28e551298c8a admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:56:17.495765 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bcd775-3480-4d2a-8cd5-28e551298c8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:56:17.496389 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bcd775-3480-4d2a-8cd5-28e551298c8a admin admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:56:17.497513 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bcd775-3480-4d2a-8cd5-28e551298c8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:56:17.501457 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bcd775-3480-4d2a-8cd5-28e551298c8a admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:56:17.518080 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 593/1176] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 11:56:17 2022] GET /identity/v3/roles => generated 1420 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 11:56:17.522583 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c332bbed-c1ac-4dee-baa1-f2f673805020 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:56:17.524419 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-c332bbed-c1ac-4dee-baa1-f2f673805020 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:56:17.541866 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c332bbed-c1ac-4dee-baa1-f2f673805020 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:56:17.571254 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:56:17.652096 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a452832-9d63-4231-8e76-981a3ae92d25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:56:17.652703 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a452832-9d63-4231-8e76-981a3ae92d25 admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:56:17.652982 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a452832-9d63-4231-8e76-981a3ae92d25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:56:17.653249 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a452832-9d63-4231-8e76-981a3ae92d25 admin admin] PATH_INFO: `/v3/roles` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:56:17.653895 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a452832-9d63-4231-8e76-981a3ae92d25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:56:17.655838 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a452832-9d63-4231-8e76-981a3ae92d25 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:56:17.668705 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 584/1177] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 11:56:17 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 11:56:17.673098 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bcd775-3480-4d2a-8cd5-28e551298c8a admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:56:17.674669 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-f5bcd775-3480-4d2a-8cd5-28e551298c8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:56:17.709355 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bcd775-3480-4d2a-8cd5-28e551298c8a admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:56:17.818726 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ec6c99-f5b1-4fd7-9962-0b5c4f50b675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:56:17.819426 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ec6c99-f5b1-4fd7-9962-0b5c4f50b675 admin admin] REQUEST_METHOD: `PUT` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:56:17.819746 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ec6c99-f5b1-4fd7-9962-0b5c4f50b675 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:56:17.820017 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ec6c99-f5b1-4fd7-9962-0b5c4f50b675 admin admin] PATH_INFO: `/v3/projects/2350ed113cb945218f9f01ee92ab0642/users/8e68191ad05744afa8254cfc63ecb396/roles/de2d2bd26fff4748a79990f97a107164` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:56:17.846080 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ec6c99-f5b1-4fd7-9962-0b5c4f50b675 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2350ed113cb945218f9f01ee92ab0642, user_id=8e68191ad05744afa8254cfc63ecb396, role_id=de2d2bd26fff4748a79990f97a107164)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:56:17.848980 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ec6c99-f5b1-4fd7-9962-0b5c4f50b675 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:56:17.877797 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 594/1178] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 11:56:17 2022] PUT /identity/v3/projects/2350ed113cb945218f9f01ee92ab0642/users/8e68191ad05744afa8254cfc63ecb396/roles/de2d2bd26fff4748a79990f97a107164 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 11:56:17.890226 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c756411-b9d5-42ba-94fd-a4bf7eb11679 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:56:17.890529 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c756411-b9d5-42ba-94fd-a4bf7eb11679 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:56:17.891248 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c756411-b9d5-42ba-94fd-a4bf7eb11679 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:56:17.952194 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-1c756411-b9d5-42ba-94fd-a4bf7eb11679 None None] MFA Rules not processed for user `8e68191ad05744afa8254cfc63ecb396`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:56:17.953407 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-1c756411-b9d5-42ba-94fd-a4bf7eb11679 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:56:18.041565 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 585/1179] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 11:56:17 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 11:56:18.055197 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97e02bc-54e4-4e4c-9bc1-4b1f2c91e172 None None] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:56:18.055481 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97e02bc-54e4-4e4c-9bc1-4b1f2c91e172 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:56:18.055752 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97e02bc-54e4-4e4c-9bc1-4b1f2c91e172 None None] PATH_INFO: `/` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:56:18.057102 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 595/1180] 10.222.0.59 () {54 vars in 963 bytes} [Tue Oct 11 11:56:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:56:18.062748 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c9bd8c-7e3f-4645-b0d9-b021f37bb794 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:56:18.063053 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c9bd8c-7e3f-4645-b0d9-b021f37bb794 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:56:18.063330 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c9bd8c-7e3f-4645-b0d9-b021f37bb794 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:56:18.096154 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:56:18.144187 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-46c9bd8c-7e3f-4645-b0d9-b021f37bb794 None None] MFA Rules not processed for user `2b1b651c73664d08accc6edf2f79bace`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:56:18.145419 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-46c9bd8c-7e3f-4645-b0d9-b021f37bb794 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:56:18.245647 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 586/1181] 10.222.0.59 () {58 vars in 1060 bytes} [Tue Oct 11 11:56:18 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 11:56:18.255508 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97e02bc-54e4-4e4c-9bc1-4b1f2c91e172 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:56:18.257056 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-f97e02bc-54e4-4e4c-9bc1-4b1f2c91e172 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:56:18.312896 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:56:18.370675 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97e02bc-54e4-4e4c-9bc1-4b1f2c91e172 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:56:18.447855 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb65f7c-d369-451c-bc38-def06e38c90d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b1b651c73664d08accc6edf2f79bace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:56:18.448469 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb65f7c-d369-451c-bc38-def06e38c90d service cinder] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:56:18.448753 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb65f7c-d369-451c-bc38-def06e38c90d service cinder] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:56:18.448999 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb65f7c-d369-451c-bc38-def06e38c90d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:56:18.450667 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-5cb65f7c-d369-451c-bc38-def06e38c90d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:56:18.553196 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb65f7c-d369-451c-bc38-def06e38c90d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:56:18.554676 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb65f7c-d369-451c-bc38-def06e38c90d service cinder] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:56:18.558228 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:56:18.573639 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 596/1182] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 11:56:18 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 11:56:18.620232 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f6c7a8-6d75-42cd-91d2-64957342ae54 None None] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:56:18.620778 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f6c7a8-6d75-42cd-91d2-64957342ae54 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:56:18.620778 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f6c7a8-6d75-42cd-91d2-64957342ae54 None None] PATH_INFO: `/` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:56:18.622147 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 587/1183] 10.222.0.59 () {52 vars in 830 bytes} [Tue Oct 11 11:56:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:56:18.626250 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb65f7c-d369-451c-bc38-def06e38c90d service cinder] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:56:18.627506 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-5cb65f7c-d369-451c-bc38-def06e38c90d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:56:18.732013 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb65f7c-d369-451c-bc38-def06e38c90d service cinder] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:56:18.832593 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98305579-7c9e-406c-9598-0858f86f99c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:56:18.833208 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98305579-7c9e-406c-9598-0858f86f99c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:56:18.833480 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98305579-7c9e-406c-9598-0858f86f99c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:56:18.833747 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98305579-7c9e-406c-9598-0858f86f99c6 admin admin] PATH_INFO: `/v3/users/8e68191ad05744afa8254cfc63ecb396` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:56:18.848429 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98305579-7c9e-406c-9598-0858f86f99c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e68191ad05744afa8254cfc63ecb396)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:56:18.849478 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: WARNING py.warnings [None req-98305579-7c9e-406c-9598-0858f86f99c6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 11 11:56:18.849478 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: warnings.warn(msg) Oct 11 11:56:18.849478 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: Oct 11 11:56:18.850734 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98305579-7c9e-406c-9598-0858f86f99c6 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:56:18.861918 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:56:18.937659 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-98305579-7c9e-406c-9598-0858f86f99c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e68191ad05744afa8254cfc63ecb396'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 11:56:18.972034 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-98305579-7c9e-406c-9598-0858f86f99c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e68191ad05744afa8254cfc63ecb396'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 11:56:18.972819 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-98305579-7c9e-406c-9598-0858f86f99c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e68191ad05744afa8254cfc63ecb396'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 11:56:18.984093 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-98305579-7c9e-406c-9598-0858f86f99c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e68191ad05744afa8254cfc63ecb396'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 11:56:18.984656 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-98305579-7c9e-406c-9598-0858f86f99c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e68191ad05744afa8254cfc63ecb396'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 11:56:19.005811 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 597/1184] 10.222.0.59 () {58 vars in 1224 bytes} [Tue Oct 11 11:56:18 2022] DELETE /identity/v3/users/8e68191ad05744afa8254cfc63ecb396 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 11:56:19.010726 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b023a9-146e-49b9-8140-35a6c762aa8d None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:56:19.011028 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b023a9-146e-49b9-8140-35a6c762aa8d None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:56:19.011386 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b023a9-146e-49b9-8140-35a6c762aa8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:56:19.074095 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-98b023a9-146e-49b9-8140-35a6c762aa8d None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:56:19.075365 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-98b023a9-146e-49b9-8140-35a6c762aa8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:56:19.168856 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:56:19.184525 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 588/1185] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 11:56:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 11:56:19.195521 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98305579-7c9e-406c-9598-0858f86f99c6 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:56:19.196942 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-98305579-7c9e-406c-9598-0858f86f99c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:56:19.309654 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98305579-7c9e-406c-9598-0858f86f99c6 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:56:19.367932 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:56:19.388275 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70e3fe4-3298-4c4a-afe6-d84b84ede1a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e563fe174db4cc19ae6bf8162cc3d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:56:19.388888 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70e3fe4-3298-4c4a-afe6-d84b84ede1a0 service neutron] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:56:19.389165 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70e3fe4-3298-4c4a-afe6-d84b84ede1a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:56:19.389434 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70e3fe4-3298-4c4a-afe6-d84b84ede1a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:56:19.391116 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-d70e3fe4-3298-4c4a-afe6-d84b84ede1a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:56:19.510730 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70e3fe4-3298-4c4a-afe6-d84b84ede1a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:56:19.512146 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70e3fe4-3298-4c4a-afe6-d84b84ede1a0 service neutron] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:56:19.532282 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 598/1186] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 11:56:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 11:56:19.910545 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b023a9-146e-49b9-8140-35a6c762aa8d None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:56:19.911834 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-98b023a9-146e-49b9-8140-35a6c762aa8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:56:19.926477 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b023a9-146e-49b9-8140-35a6c762aa8d None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:56:20.038365 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0371acd-70ea-4b99-bd2d-15013bac8227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:56:20.038944 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0371acd-70ea-4b99-bd2d-15013bac8227 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:56:20.039222 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0371acd-70ea-4b99-bd2d-15013bac8227 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:56:20.039487 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0371acd-70ea-4b99-bd2d-15013bac8227 admin admin] PATH_INFO: `/v3/projects/2350ed113cb945218f9f01ee92ab0642` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:56:20.050573 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0371acd-70ea-4b99-bd2d-15013bac8227 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2350ed113cb945218f9f01ee92ab0642)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:56:20.051699 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: WARNING py.warnings [None req-a0371acd-70ea-4b99-bd2d-15013bac8227 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 11 11:56:20.051699 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: warnings.warn(msg) Oct 11 11:56:20.051699 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: Oct 11 11:56:20.052933 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0371acd-70ea-4b99-bd2d-15013bac8227 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:56:20.123691 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-a0371acd-70ea-4b99-bd2d-15013bac8227 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2350ed113cb945218f9f01ee92ab0642', 'request_id': 'req-a0371acd-70ea-4b99-bd2d-15013bac8227'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 11:56:20.131237 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-a0371acd-70ea-4b99-bd2d-15013bac8227 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2350ed113cb945218f9f01ee92ab0642', 'request_id': 'req-a0371acd-70ea-4b99-bd2d-15013bac8227'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 11:56:20.142428 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-a0371acd-70ea-4b99-bd2d-15013bac8227 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2350ed113cb945218f9f01ee92ab0642', 'request_id': 'req-a0371acd-70ea-4b99-bd2d-15013bac8227'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 11:56:20.177389 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-a0371acd-70ea-4b99-bd2d-15013bac8227 admin admin] The token cache is being invalidate because project 2350ed113cb945218f9f01ee92ab0642 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 11:56:20.177717 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-a0371acd-70ea-4b99-bd2d-15013bac8227 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 11:56:20.178162 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-a0371acd-70ea-4b99-bd2d-15013bac8227 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 11:56:20.179665 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 589/1187] 10.222.0.59 () {58 vars in 1233 bytes} [Tue Oct 11 11:56:19 2022] DELETE /identity/v3/projects/2350ed113cb945218f9f01ee92ab0642 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 11:56:29.529566 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=62 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:56:30.175035 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:57:50.593324 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81e50fe-251e-46fd-a9cc-a83d6c1010a8 None None] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:57:50.593979 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81e50fe-251e-46fd-a9cc-a83d6c1010a8 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:57:50.593979 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81e50fe-251e-46fd-a9cc-a83d6c1010a8 None None] PATH_INFO: `/` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:57:50.595891 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 599/1188] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 11 11:57:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:57:50.602105 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3324000-329e-4a16-b20e-5e47f5e13e34 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:57:50.602671 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3324000-329e-4a16-b20e-5e47f5e13e34 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:57:50.602671 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3324000-329e-4a16-b20e-5e47f5e13e34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:57:50.722895 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-d3324000-329e-4a16-b20e-5e47f5e13e34 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:57:50.724149 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-d3324000-329e-4a16-b20e-5e47f5e13e34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:57:50.840040 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 590/1189] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 11 11:57:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 11:57:50.993458 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81e50fe-251e-46fd-a9cc-a83d6c1010a8 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:57:50.996697 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-c81e50fe-251e-46fd-a9cc-a83d6c1010a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:57:51.132563 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81e50fe-251e-46fd-a9cc-a83d6c1010a8 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:57:51.138974 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c814b474-0970-4848-81a8-a5691c6efd80 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bad5e1cb306b46818096940acc3810b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:57:51.139608 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c814b474-0970-4848-81a8-a5691c6efd80 service nova] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:57:51.139937 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c814b474-0970-4848-81a8-a5691c6efd80 service nova] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:57:51.140223 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c814b474-0970-4848-81a8-a5691c6efd80 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:57:51.148802 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c814b474-0970-4848-81a8-a5691c6efd80 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:57:51.150592 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c814b474-0970-4848-81a8-a5691c6efd80 service nova] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:57:51.160326 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 600/1190] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 11:57:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 11:57:52.247761 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bdd9af-2269-4849-9000-dcbd038f4cd9 None None] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:57:52.248313 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bdd9af-2269-4849-9000-dcbd038f4cd9 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:57:52.248313 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bdd9af-2269-4849-9000-dcbd038f4cd9 None None] PATH_INFO: `/` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:57:52.250219 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 591/1191] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 11 11:57:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:57:52.263946 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99a4257-6f05-4c9c-b388-02a21e65dea2 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:57:52.264577 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99a4257-6f05-4c9c-b388-02a21e65dea2 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:57:52.264577 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99a4257-6f05-4c9c-b388-02a21e65dea2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:57:52.295197 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-e99a4257-6f05-4c9c-b388-02a21e65dea2 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:57:52.296983 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-e99a4257-6f05-4c9c-b388-02a21e65dea2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:57:52.310904 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 601/1192] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 11 11:57:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 11:57:52.467605 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bdd9af-2269-4849-9000-dcbd038f4cd9 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:57:52.489409 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bdd9af-2269-4849-9000-dcbd038f4cd9 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:57:52.497983 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dd134e-3d02-4a55-833f-b90119b41f21 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bad5e1cb306b46818096940acc3810b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:57:52.498580 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dd134e-3d02-4a55-833f-b90119b41f21 service nova] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:57:52.498931 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dd134e-3d02-4a55-833f-b90119b41f21 service nova] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:57:52.499211 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dd134e-3d02-4a55-833f-b90119b41f21 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:57:52.508137 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dd134e-3d02-4a55-833f-b90119b41f21 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:57:52.509782 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dd134e-3d02-4a55-833f-b90119b41f21 service nova] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:57:52.520354 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 592/1193] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 11:57:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 11:57:52.560611 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0195ff31-c617-4635-bd41-74c447d73fa7 None None] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:57:52.560880 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0195ff31-c617-4635-bd41-74c447d73fa7 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:57:52.561159 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0195ff31-c617-4635-bd41-74c447d73fa7 None None] PATH_INFO: `/v3` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:57:52.562484 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 602/1194] 10.222.0.50 () {54 vars in 925 bytes} [Tue Oct 11 11:57:52 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:57:52.569657 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4560d19-70e3-4c94-b914-7ee6bb7269c7 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:57:52.570026 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4560d19-70e3-4c94-b914-7ee6bb7269c7 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:57:52.570201 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4560d19-70e3-4c94-b914-7ee6bb7269c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:57:52.643347 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-c4560d19-70e3-4c94-b914-7ee6bb7269c7 None None] MFA Rules not processed for user `f272cdd1bcc04b4e8261ed533cfd4fd7`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:57:52.644616 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-c4560d19-70e3-4c94-b914-7ee6bb7269c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:57:52.664299 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:57:52.733725 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 593/1195] 10.222.0.50 () {58 vars in 1015 bytes} [Tue Oct 11 11:57:52 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 11:58:01.770584 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fd2bdc-95b1-4d31-ab55-2cc5dc638d5c None None] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:58:01.771038 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fd2bdc-95b1-4d31-ab55-2cc5dc638d5c None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:58:01.771155 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fd2bdc-95b1-4d31-ab55-2cc5dc638d5c None None] PATH_INFO: `/` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:58:01.772463 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 603/1196] 10.222.0.59 () {54 vars in 961 bytes} [Tue Oct 11 11:58:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:58:01.777644 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621e7fcb-b854-4ad1-9ec2-80c66fa05f7e None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:58:01.778131 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621e7fcb-b854-4ad1-9ec2-80c66fa05f7e None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:58:01.778243 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621e7fcb-b854-4ad1-9ec2-80c66fa05f7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:58:01.799015 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-621e7fcb-b854-4ad1-9ec2-80c66fa05f7e None None] MFA Rules not processed for user `f272cdd1bcc04b4e8261ed533cfd4fd7`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:58:01.800195 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-621e7fcb-b854-4ad1-9ec2-80c66fa05f7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:58:01.809781 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 594/1197] 10.222.0.59 () {58 vars in 1058 bytes} [Tue Oct 11 11:58:01 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 11:58:01.817595 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fd2bdc-95b1-4d31-ab55-2cc5dc638d5c None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:58:01.834238 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fd2bdc-95b1-4d31-ab55-2cc5dc638d5c None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:58:01.840126 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19045d5-5112-4be2-81f6-4155a58ba1b5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f272cdd1bcc04b4e8261ed533cfd4fd7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:58:01.840766 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19045d5-5112-4be2-81f6-4155a58ba1b5 service placement] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:58:01.841312 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19045d5-5112-4be2-81f6-4155a58ba1b5 service placement] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:58:01.841719 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19045d5-5112-4be2-81f6-4155a58ba1b5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:58:01.854922 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19045d5-5112-4be2-81f6-4155a58ba1b5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:58:01.857233 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19045d5-5112-4be2-81f6-4155a58ba1b5 service placement] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:58:01.867490 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 604/1198] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 11:58:01 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 11:58:04.000500 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf1aac7-0db4-40ac-b21d-26b170ac8aee None None] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:58:04.001077 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf1aac7-0db4-40ac-b21d-26b170ac8aee None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:58:04.001077 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf1aac7-0db4-40ac-b21d-26b170ac8aee None None] PATH_INFO: `/` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:58:04.002431 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 595/1199] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 11 11:58:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:58:04.009266 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2004af9d-5a51-4814-b8e4-fefad24f2a18 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:58:04.009747 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2004af9d-5a51-4814-b8e4-fefad24f2a18 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:58:04.009871 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2004af9d-5a51-4814-b8e4-fefad24f2a18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:58:04.031821 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-2004af9d-5a51-4814-b8e4-fefad24f2a18 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:58:04.033055 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-2004af9d-5a51-4814-b8e4-fefad24f2a18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:58:04.042428 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 605/1200] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 11 11:58:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 11:58:04.201475 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf1aac7-0db4-40ac-b21d-26b170ac8aee None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:58:04.218111 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf1aac7-0db4-40ac-b21d-26b170ac8aee None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:58:04.226370 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457837e0-1046-4247-89d5-a5c066ff07d8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bad5e1cb306b46818096940acc3810b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:58:04.226999 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457837e0-1046-4247-89d5-a5c066ff07d8 service nova] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:58:04.227282 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457837e0-1046-4247-89d5-a5c066ff07d8 service nova] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:58:04.227549 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457837e0-1046-4247-89d5-a5c066ff07d8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:58:04.237151 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457837e0-1046-4247-89d5-a5c066ff07d8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:58:04.238793 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457837e0-1046-4247-89d5-a5c066ff07d8 service nova] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:58:04.248924 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 596/1201] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 11:58:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 11:58:05.320745 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0901a5d2-06a4-4f7d-8f25-f43925d5a4c7 None None] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:58:05.321282 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0901a5d2-06a4-4f7d-8f25-f43925d5a4c7 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:58:05.321282 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0901a5d2-06a4-4f7d-8f25-f43925d5a4c7 None None] PATH_INFO: `/` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:58:05.322721 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 606/1202] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 11 11:58:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:58:05.330514 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4322cb40-e4f5-4a19-9adf-5b0ffff784e4 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:58:05.331316 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4322cb40-e4f5-4a19-9adf-5b0ffff784e4 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:58:05.331422 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4322cb40-e4f5-4a19-9adf-5b0ffff784e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:58:05.355773 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-4322cb40-e4f5-4a19-9adf-5b0ffff784e4 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:58:05.356987 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-4322cb40-e4f5-4a19-9adf-5b0ffff784e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:58:05.366116 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 597/1203] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 11 11:58:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 11:58:05.527544 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0901a5d2-06a4-4f7d-8f25-f43925d5a4c7 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:58:05.550296 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0901a5d2-06a4-4f7d-8f25-f43925d5a4c7 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:58:05.556449 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9579090f-bf55-4800-b1d4-3b51d930fbda service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bad5e1cb306b46818096940acc3810b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:58:05.557030 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9579090f-bf55-4800-b1d4-3b51d930fbda service nova] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:58:05.557309 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9579090f-bf55-4800-b1d4-3b51d930fbda service nova] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:58:05.557586 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9579090f-bf55-4800-b1d4-3b51d930fbda service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:58:05.565924 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9579090f-bf55-4800-b1d4-3b51d930fbda service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:58:05.567551 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9579090f-bf55-4800-b1d4-3b51d930fbda service nova] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:58:05.577345 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 607/1204] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 11:58:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 11:58:09.627072 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a72c203-ae94-40f7-9d7a-93a83d9f25e1 None None] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:58:09.627624 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a72c203-ae94-40f7-9d7a-93a83d9f25e1 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:58:09.627624 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a72c203-ae94-40f7-9d7a-93a83d9f25e1 None None] PATH_INFO: `/` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:58:09.628887 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 598/1205] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 11 11:58:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:58:09.635481 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d5c6f9-2a20-417b-8c37-bb6d9a8f9ed3 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:58:09.635948 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d5c6f9-2a20-417b-8c37-bb6d9a8f9ed3 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:58:09.636058 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d5c6f9-2a20-417b-8c37-bb6d9a8f9ed3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:58:09.657924 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-c6d5c6f9-2a20-417b-8c37-bb6d9a8f9ed3 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:58:09.659177 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-c6d5c6f9-2a20-417b-8c37-bb6d9a8f9ed3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:58:09.669156 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 608/1206] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 11 11:58:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 11:58:09.954269 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a72c203-ae94-40f7-9d7a-93a83d9f25e1 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:58:09.956472 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-6a72c203-ae94-40f7-9d7a-93a83d9f25e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:58:10.044943 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a72c203-ae94-40f7-9d7a-93a83d9f25e1 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:58:10.049512 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9103936-d1a8-485a-a3f1-85bc312112b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e563fe174db4cc19ae6bf8162cc3d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:58:10.050121 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9103936-d1a8-485a-a3f1-85bc312112b7 service neutron] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:58:10.050396 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9103936-d1a8-485a-a3f1-85bc312112b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:58:10.050696 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9103936-d1a8-485a-a3f1-85bc312112b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:58:10.061019 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9103936-d1a8-485a-a3f1-85bc312112b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:58:10.062300 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9103936-d1a8-485a-a3f1-85bc312112b7 service neutron] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:58:10.072055 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 599/1207] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 11:58:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 11:58:12.203997 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a28017f-0ab2-4aa1-ac32-89d1ed2a3fee None None] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:58:12.204604 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a28017f-0ab2-4aa1-ac32-89d1ed2a3fee None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:58:12.204604 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a28017f-0ab2-4aa1-ac32-89d1ed2a3fee None None] PATH_INFO: `/` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:58:12.205900 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 609/1208] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 11 11:58:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:58:12.211888 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67db90f1-f55f-483b-99fb-c825524a044c None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:58:12.212321 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67db90f1-f55f-483b-99fb-c825524a044c None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:58:12.212450 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67db90f1-f55f-483b-99fb-c825524a044c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:58:12.233931 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-67db90f1-f55f-483b-99fb-c825524a044c None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:58:12.235119 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-67db90f1-f55f-483b-99fb-c825524a044c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:58:12.243509 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 600/1209] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 11 11:58:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 11:58:12.525425 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a28017f-0ab2-4aa1-ac32-89d1ed2a3fee None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:58:12.537897 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a28017f-0ab2-4aa1-ac32-89d1ed2a3fee None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:58:12.542010 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ab1251-4ed9-4b6a-9fc5-392dfe610b3c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e563fe174db4cc19ae6bf8162cc3d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:58:12.542695 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ab1251-4ed9-4b6a-9fc5-392dfe610b3c service neutron] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:58:12.542970 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ab1251-4ed9-4b6a-9fc5-392dfe610b3c service neutron] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:58:12.543239 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ab1251-4ed9-4b6a-9fc5-392dfe610b3c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:58:12.552881 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ab1251-4ed9-4b6a-9fc5-392dfe610b3c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:58:12.554322 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ab1251-4ed9-4b6a-9fc5-392dfe610b3c service neutron] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:58:12.563781 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 610/1210] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 11:58:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 11:58:16.218832 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f5fd27-b554-4c0c-a285-d6a1b11f39f9 None None] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:58:16.219373 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f5fd27-b554-4c0c-a285-d6a1b11f39f9 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:58:16.219373 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f5fd27-b554-4c0c-a285-d6a1b11f39f9 None None] PATH_INFO: `/` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:58:16.220783 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 601/1211] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 11 11:58:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:58:16.226472 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95ed66b-77c6-4c6d-922a-a08bc140fb66 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:58:16.227010 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95ed66b-77c6-4c6d-922a-a08bc140fb66 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:58:16.227010 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95ed66b-77c6-4c6d-922a-a08bc140fb66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:58:16.248268 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-e95ed66b-77c6-4c6d-922a-a08bc140fb66 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:58:16.249500 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-e95ed66b-77c6-4c6d-922a-a08bc140fb66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:58:16.258072 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 611/1212] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 11 11:58:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 11:58:16.558654 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f5fd27-b554-4c0c-a285-d6a1b11f39f9 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:58:16.561346 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-44f5fd27-b554-4c0c-a285-d6a1b11f39f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:58:16.628194 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:58:16.643037 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f5fd27-b554-4c0c-a285-d6a1b11f39f9 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:58:16.647662 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edd5ffe-080a-4c84-a342-dbc8600f98cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b1b651c73664d08accc6edf2f79bace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:58:16.648300 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edd5ffe-080a-4c84-a342-dbc8600f98cd service cinder] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:58:16.648576 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edd5ffe-080a-4c84-a342-dbc8600f98cd service cinder] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:58:16.648871 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edd5ffe-080a-4c84-a342-dbc8600f98cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:58:16.659368 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edd5ffe-080a-4c84-a342-dbc8600f98cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:58:16.660805 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edd5ffe-080a-4c84-a342-dbc8600f98cd service cinder] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:58:16.670701 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 602/1213] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 11:58:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 11:58:18.841691 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245532c4-fb0e-496d-982a-081d93691073 None None] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:58:18.842441 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245532c4-fb0e-496d-982a-081d93691073 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:58:18.842441 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245532c4-fb0e-496d-982a-081d93691073 None None] PATH_INFO: `/` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:58:18.843643 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 612/1214] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 11 11:58:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:58:18.849110 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c262d14-6b06-4c7c-af7d-bcfd6b4810c3 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:58:18.849606 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c262d14-6b06-4c7c-af7d-bcfd6b4810c3 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:58:18.849606 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c262d14-6b06-4c7c-af7d-bcfd6b4810c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:58:18.871786 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-6c262d14-6b06-4c7c-af7d-bcfd6b4810c3 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:58:18.873127 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-6c262d14-6b06-4c7c-af7d-bcfd6b4810c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:58:18.881948 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 603/1215] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 11 11:58:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 11:58:19.177852 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245532c4-fb0e-496d-982a-081d93691073 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:58:19.179976 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-245532c4-fb0e-496d-982a-081d93691073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:58:19.194320 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245532c4-fb0e-496d-982a-081d93691073 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:58:19.198202 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f942e6-a9a5-47e6-a0e5-caf61b4bf0f6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b1b651c73664d08accc6edf2f79bace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:58:19.198830 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f942e6-a9a5-47e6-a0e5-caf61b4bf0f6 service cinder] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:58:19.199106 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f942e6-a9a5-47e6-a0e5-caf61b4bf0f6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:58:19.199369 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f942e6-a9a5-47e6-a0e5-caf61b4bf0f6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:58:19.209362 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f942e6-a9a5-47e6-a0e5-caf61b4bf0f6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:58:19.210823 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f942e6-a9a5-47e6-a0e5-caf61b4bf0f6 service cinder] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:58:19.220192 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 613/1216] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 11:58:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 11:58:28.866986 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=11 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:58:29.216228 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=89 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:59:44.177062 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82399b0e-8065-4271-b0f2-d7074a99cbe3 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:44.177747 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82399b0e-8065-4271-b0f2-d7074a99cbe3 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:44.177747 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82399b0e-8065-4271-b0f2-d7074a99cbe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:44.211880 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-82399b0e-8065-4271-b0f2-d7074a99cbe3 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:59:44.213322 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-82399b0e-8065-4271-b0f2-d7074a99cbe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:59:44.222690 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 604/1217] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 11:59:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 11:59:44.233993 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247190c4-91f8-4a5f-8405-e2c99f978b65 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:44.234649 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247190c4-91f8-4a5f-8405-e2c99f978b65 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:44.234649 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247190c4-91f8-4a5f-8405-e2c99f978b65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:44.265434 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-247190c4-91f8-4a5f-8405-e2c99f978b65 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:59:44.267277 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-247190c4-91f8-4a5f-8405-e2c99f978b65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:59:44.276052 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 614/1218] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 11:59:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 11:59:44.281713 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82399b0e-8065-4271-b0f2-d7074a99cbe3 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:59:44.296021 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82399b0e-8065-4271-b0f2-d7074a99cbe3 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:59:44.305345 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c5290e-50a1-4868-b01d-5f67b0e6c711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:59:44.305978 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c5290e-50a1-4868-b01d-5f67b0e6c711 admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:44.306253 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c5290e-50a1-4868-b01d-5f67b0e6c711 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:44.306519 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c5290e-50a1-4868-b01d-5f67b0e6c711 admin admin] PATH_INFO: `/v3/domains` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:44.307199 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c5290e-50a1-4868-b01d-5f67b0e6c711 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:59:44.307819 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c5290e-50a1-4868-b01d-5f67b0e6c711 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:59:44.309433 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c5290e-50a1-4868-b01d-5f67b0e6c711 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:59:44.326842 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 605/1219] 10.222.0.59 () {58 vars in 1153 bytes} [Tue Oct 11 11:59:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:59:44.336347 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247190c4-91f8-4a5f-8405-e2c99f978b65 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:59:44.352148 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247190c4-91f8-4a5f-8405-e2c99f978b65 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:59:44.360328 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9cee2d-e430-4570-87f7-6b978716f11d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:59:44.360989 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9cee2d-e430-4570-87f7-6b978716f11d admin admin] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:44.361264 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9cee2d-e430-4570-87f7-6b978716f11d admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:44.361531 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9cee2d-e430-4570-87f7-6b978716f11d admin admin] PATH_INFO: `/v3/projects` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:44.362193 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9cee2d-e430-4570-87f7-6b978716f11d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:59:44.364333 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9cee2d-e430-4570-87f7-6b978716f11d admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:59:44.424336 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 615/1220] 10.222.0.59 () {60 vars in 1153 bytes} [Tue Oct 11 11:59:44 2022] POST /identity/v3/projects => generated 385 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 11:59:44.428676 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c5290e-50a1-4868-b01d-5f67b0e6c711 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:59:44.444538 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c5290e-50a1-4868-b01d-5f67b0e6c711 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:59:44.482427 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6406f4c-5578-49b7-b7ff-a4fd2f146e8d None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:44.482669 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6406f4c-5578-49b7-b7ff-a4fd2f146e8d None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:44.482906 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6406f4c-5578-49b7-b7ff-a4fd2f146e8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:44.520751 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-c6406f4c-5578-49b7-b7ff-a4fd2f146e8d None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:59:44.521982 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-c6406f4c-5578-49b7-b7ff-a4fd2f146e8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:59:44.561398 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979c0a04-49a5-4359-80a6-88f6a50719cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:59:44.562017 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979c0a04-49a5-4359-80a6-88f6a50719cc admin admin] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:44.562295 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979c0a04-49a5-4359-80a6-88f6a50719cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:44.562581 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979c0a04-49a5-4359-80a6-88f6a50719cc admin admin] PATH_INFO: `/v3/users` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:44.563244 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979c0a04-49a5-4359-80a6-88f6a50719cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:59:44.567590 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979c0a04-49a5-4359-80a6-88f6a50719cc admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:59:44.602030 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 606/1221] 10.222.0.59 () {60 vars in 1144 bytes} [Tue Oct 11 11:59:44 2022] POST /identity/v3/users => generated 293 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 11:59:44.603647 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50b3c2a-de1f-4cb3-b267-35a3fe71c3bf None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:44.604313 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50b3c2a-de1f-4cb3-b267-35a3fe71c3bf None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:44.604602 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50b3c2a-de1f-4cb3-b267-35a3fe71c3bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:44.627602 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-d50b3c2a-de1f-4cb3-b267-35a3fe71c3bf None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:59:44.628761 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 616/1222] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 11:59:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 11:59:44.628893 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-d50b3c2a-de1f-4cb3-b267-35a3fe71c3bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:59:44.630053 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c0106d-5c5d-447e-b9c9-d83aebb14bbe None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:44.630338 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c0106d-5c5d-447e-b9c9-d83aebb14bbe None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:44.630639 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c0106d-5c5d-447e-b9c9-d83aebb14bbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:44.641808 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 607/1223] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 11:59:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 11:59:44.643594 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99521a2-b575-4758-90e1-31315da639f3 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:44.643883 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99521a2-b575-4758-90e1-31315da639f3 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:44.644327 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99521a2-b575-4758-90e1-31315da639f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:44.655528 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-67c0106d-5c5d-447e-b9c9-d83aebb14bbe None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:59:44.656753 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-67c0106d-5c5d-447e-b9c9-d83aebb14bbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:59:44.665279 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 617/1224] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 11:59:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 11:59:44.665673 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-d99521a2-b575-4758-90e1-31315da639f3 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:59:44.666535 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1ba458-8b4f-4fcc-b355-cabf5c4fc18e None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:44.666859 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1ba458-8b4f-4fcc-b355-cabf5c4fc18e None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:44.666990 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-d99521a2-b575-4758-90e1-31315da639f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:59:44.667264 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1ba458-8b4f-4fcc-b355-cabf5c4fc18e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:44.675956 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 608/1225] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 11:59:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 11:59:44.677351 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef948d2-f074-4443-961e-cac2be893b6b None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:44.677639 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef948d2-f074-4443-961e-cac2be893b6b None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:44.677918 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef948d2-f074-4443-961e-cac2be893b6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:44.688479 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-2c1ba458-8b4f-4fcc-b355-cabf5c4fc18e None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:59:44.689707 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-2c1ba458-8b4f-4fcc-b355-cabf5c4fc18e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:59:44.697964 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 618/1226] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 11:59:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 11:59:44.698945 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1ba458-8b4f-4fcc-b355-cabf5c4fc18e None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:59:44.699653 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-aef948d2-f074-4443-961e-cac2be893b6b None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:59:44.700873 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-aef948d2-f074-4443-961e-cac2be893b6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:59:44.709386 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1ba458-8b4f-4fcc-b355-cabf5c4fc18e None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:59:44.710459 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 609/1227] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 11:59:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 11:59:44.712010 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722d1d77-5fe6-4905-acfe-d6c1deba9b9e None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:44.712194 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722d1d77-5fe6-4905-acfe-d6c1deba9b9e None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:44.712359 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722d1d77-5fe6-4905-acfe-d6c1deba9b9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:44.715559 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220394dd-1ad8-4e37-839b-09934936d72c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:59:44.716232 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220394dd-1ad8-4e37-839b-09934936d72c admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:44.716511 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220394dd-1ad8-4e37-839b-09934936d72c admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:44.716787 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220394dd-1ad8-4e37-839b-09934936d72c admin admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:44.717454 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220394dd-1ad8-4e37-839b-09934936d72c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:59:44.719157 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220394dd-1ad8-4e37-839b-09934936d72c admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:59:44.730733 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 619/1228] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 11:59:44 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 11:59:44.734027 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef131d1-0a7f-405a-bc16-95494f85baca None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:44.734027 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef131d1-0a7f-405a-bc16-95494f85baca None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:44.734027 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef131d1-0a7f-405a-bc16-95494f85baca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:44.735001 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-722d1d77-5fe6-4905-acfe-d6c1deba9b9e None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:59:44.736241 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-722d1d77-5fe6-4905-acfe-d6c1deba9b9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:59:44.744337 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 610/1229] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 11:59:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 11:59:44.745662 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851d01a6-8040-47ac-84f9-b9664135ff6a None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:44.745945 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851d01a6-8040-47ac-84f9-b9664135ff6a None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:44.746222 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851d01a6-8040-47ac-84f9-b9664135ff6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:44.759061 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-8ef131d1-0a7f-405a-bc16-95494f85baca None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:59:44.760383 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-8ef131d1-0a7f-405a-bc16-95494f85baca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:59:44.769252 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 620/1230] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 11:59:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 11:59:44.770048 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-851d01a6-8040-47ac-84f9-b9664135ff6a None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:59:44.770694 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5e6979-1566-47a3-88da-2919d14b9a0c None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:44.771051 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5e6979-1566-47a3-88da-2919d14b9a0c None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:44.771311 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-851d01a6-8040-47ac-84f9-b9664135ff6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:59:44.771431 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5e6979-1566-47a3-88da-2919d14b9a0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:44.783134 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 611/1231] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 11:59:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 11:59:44.783134 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2b3c84-5a94-4e6c-9f39-875dbebe11fb None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:44.783134 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2b3c84-5a94-4e6c-9f39-875dbebe11fb None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:44.783134 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2b3c84-5a94-4e6c-9f39-875dbebe11fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:44.797156 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-8e5e6979-1566-47a3-88da-2919d14b9a0c None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:59:44.798386 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-8e5e6979-1566-47a3-88da-2919d14b9a0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:59:44.801588 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-5f2b3c84-5a94-4e6c-9f39-875dbebe11fb None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:59:44.802761 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-5f2b3c84-5a94-4e6c-9f39-875dbebe11fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:59:44.807803 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 621/1232] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 11:59:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 11:59:44.808968 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96be551-a4e8-4f18-8327-d76b9d1283dd None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:44.809219 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96be551-a4e8-4f18-8327-d76b9d1283dd None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:44.809498 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96be551-a4e8-4f18-8327-d76b9d1283dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:44.812851 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 612/1233] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 11:59:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 11:59:44.814245 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248a7497-5d56-4f34-9765-bbacabd8e21f None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:44.814531 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248a7497-5d56-4f34-9765-bbacabd8e21f None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:44.814822 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248a7497-5d56-4f34-9765-bbacabd8e21f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:44.830377 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-c96be551-a4e8-4f18-8327-d76b9d1283dd None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:59:44.831671 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-c96be551-a4e8-4f18-8327-d76b9d1283dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:59:44.835961 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-248a7497-5d56-4f34-9765-bbacabd8e21f None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:59:44.837210 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-248a7497-5d56-4f34-9765-bbacabd8e21f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:59:44.840333 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 622/1234] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 11:59:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 11:59:44.841208 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96be551-a4e8-4f18-8327-d76b9d1283dd None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:59:44.845753 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 613/1235] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 11:59:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 11:59:44.847255 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf677cf-53b7-4d63-aee8-5881be403e63 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:44.847543 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf677cf-53b7-4d63-aee8-5881be403e63 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:44.847815 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf677cf-53b7-4d63-aee8-5881be403e63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:44.855408 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96be551-a4e8-4f18-8327-d76b9d1283dd None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:59:44.860545 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66454e6-e069-498b-8bf2-e73fce24deb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:59:44.861185 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66454e6-e069-498b-8bf2-e73fce24deb6 admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:44.861464 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66454e6-e069-498b-8bf2-e73fce24deb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:44.861769 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66454e6-e069-498b-8bf2-e73fce24deb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:44.862420 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66454e6-e069-498b-8bf2-e73fce24deb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:59:44.864178 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66454e6-e069-498b-8bf2-e73fce24deb6 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:59:44.868405 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-9bf677cf-53b7-4d63-aee8-5881be403e63 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:59:44.869656 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-9bf677cf-53b7-4d63-aee8-5881be403e63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:59:44.878490 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 623/1236] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 11:59:44 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 11:59:44.878490 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66454e6-e069-498b-8bf2-e73fce24deb6 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:59:44.881551 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 614/1237] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 11:59:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 11:59:44.882469 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf677cf-53b7-4d63-aee8-5881be403e63 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:59:44.887913 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66454e6-e069-498b-8bf2-e73fce24deb6 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:59:44.892938 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e9df47-bcd0-4c60-bf8a-4b33fe396033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:59:44.893139 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf677cf-53b7-4d63-aee8-5881be403e63 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:59:44.893726 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e9df47-bcd0-4c60-bf8a-4b33fe396033 admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:44.894008 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e9df47-bcd0-4c60-bf8a-4b33fe396033 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:44.894286 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e9df47-bcd0-4c60-bf8a-4b33fe396033 admin admin] PATH_INFO: `/v3/domains` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:44.894980 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e9df47-bcd0-4c60-bf8a-4b33fe396033 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:59:44.895336 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e9df47-bcd0-4c60-bf8a-4b33fe396033 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:59:44.896368 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: WARNING py.warnings [None req-e2e9df47-bcd0-4c60-bf8a-4b33fe396033 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 11 11:59:44.896368 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: warnings.warn(msg) Oct 11 11:59:44.896368 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: Oct 11 11:59:44.897266 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e9df47-bcd0-4c60-bf8a-4b33fe396033 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:59:44.899847 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db8b174-4403-4750-934c-7d9303602bf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:59:44.900581 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db8b174-4403-4750-934c-7d9303602bf4 admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:44.900932 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db8b174-4403-4750-934c-7d9303602bf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:44.901275 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db8b174-4403-4750-934c-7d9303602bf4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:44.902000 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db8b174-4403-4750-934c-7d9303602bf4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:59:44.902381 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db8b174-4403-4750-934c-7d9303602bf4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:59:44.904484 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db8b174-4403-4750-934c-7d9303602bf4 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:59:44.910431 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 624/1238] 10.222.0.59 () {58 vars in 1153 bytes} [Tue Oct 11 11:59:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:59:44.911383 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e9df47-bcd0-4c60-bf8a-4b33fe396033 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:59:44.927528 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 615/1239] 10.222.0.59 () {58 vars in 1153 bytes} [Tue Oct 11 11:59:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:59:44.927832 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e9df47-bcd0-4c60-bf8a-4b33fe396033 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:59:44.927980 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db8b174-4403-4750-934c-7d9303602bf4 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:59:44.932920 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870106be-aec7-4626-a7cc-b89f07a0f387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:59:44.933440 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db8b174-4403-4750-934c-7d9303602bf4 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:59:44.933550 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870106be-aec7-4626-a7cc-b89f07a0f387 admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:44.933819 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870106be-aec7-4626-a7cc-b89f07a0f387 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:44.934091 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870106be-aec7-4626-a7cc-b89f07a0f387 admin admin] PATH_INFO: `/v3/domains` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:44.935005 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870106be-aec7-4626-a7cc-b89f07a0f387 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:59:44.935330 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870106be-aec7-4626-a7cc-b89f07a0f387 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:59:44.937133 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870106be-aec7-4626-a7cc-b89f07a0f387 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:59:44.939114 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d506ba-79c0-4ff4-9f94-28933d4b92d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:59:44.939708 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d506ba-79c0-4ff4-9f94-28933d4b92d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:44.939988 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d506ba-79c0-4ff4-9f94-28933d4b92d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:44.940266 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d506ba-79c0-4ff4-9f94-28933d4b92d6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:44.940934 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d506ba-79c0-4ff4-9f94-28933d4b92d6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:59:44.941238 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d506ba-79c0-4ff4-9f94-28933d4b92d6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:59:44.943042 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d506ba-79c0-4ff4-9f94-28933d4b92d6 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:59:44.949470 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 625/1240] 10.222.0.59 () {58 vars in 1153 bytes} [Tue Oct 11 11:59:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:59:44.950322 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870106be-aec7-4626-a7cc-b89f07a0f387 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:59:44.960421 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 616/1241] 10.222.0.59 () {58 vars in 1153 bytes} [Tue Oct 11 11:59:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:59:44.961297 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d506ba-79c0-4ff4-9f94-28933d4b92d6 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:59:44.962975 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870106be-aec7-4626-a7cc-b89f07a0f387 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:59:44.970410 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2890903-52a5-4349-a02f-d7c0af1fe109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:59:44.971100 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2890903-52a5-4349-a02f-d7c0af1fe109 admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:44.971378 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2890903-52a5-4349-a02f-d7c0af1fe109 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:44.971646 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2890903-52a5-4349-a02f-d7c0af1fe109 admin admin] PATH_INFO: `/v3/domains` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:44.972198 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d506ba-79c0-4ff4-9f94-28933d4b92d6 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:59:44.972318 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2890903-52a5-4349-a02f-d7c0af1fe109 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:59:44.972615 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2890903-52a5-4349-a02f-d7c0af1fe109 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:59:44.974328 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2890903-52a5-4349-a02f-d7c0af1fe109 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:59:44.978206 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3c6b7f-9194-4cae-a230-d1c4aa230a56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:59:44.978858 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3c6b7f-9194-4cae-a230-d1c4aa230a56 admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:44.979135 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3c6b7f-9194-4cae-a230-d1c4aa230a56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:44.979410 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3c6b7f-9194-4cae-a230-d1c4aa230a56 admin admin] PATH_INFO: `/v3/domains` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:44.980092 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3c6b7f-9194-4cae-a230-d1c4aa230a56 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:59:44.980404 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3c6b7f-9194-4cae-a230-d1c4aa230a56 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:59:44.982012 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3c6b7f-9194-4cae-a230-d1c4aa230a56 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:59:44.988135 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 626/1242] 10.222.0.59 () {58 vars in 1153 bytes} [Tue Oct 11 11:59:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:59:44.988916 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2890903-52a5-4349-a02f-d7c0af1fe109 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:59:44.996507 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 617/1243] 10.222.0.59 () {58 vars in 1153 bytes} [Tue Oct 11 11:59:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:59:44.997360 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3c6b7f-9194-4cae-a230-d1c4aa230a56 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:59:45.003295 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2890903-52a5-4349-a02f-d7c0af1fe109 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:59:45.007833 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3c6b7f-9194-4cae-a230-d1c4aa230a56 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:59:45.009005 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a46084b-bb0a-4db6-a2a8-40d1ab79707e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:59:45.009686 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a46084b-bb0a-4db6-a2a8-40d1ab79707e admin admin] REQUEST_METHOD: `PUT` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:45.009962 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a46084b-bb0a-4db6-a2a8-40d1ab79707e admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:45.010247 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a46084b-bb0a-4db6-a2a8-40d1ab79707e admin admin] PATH_INFO: `/v3/projects/81ed94c4abf74746bb798c6cf80173ca/users/443b625c635944189820425cea6c5722/roles/de2d2bd26fff4748a79990f97a107164` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:45.012423 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c624387-b1af-4cf1-8eda-5088c6b608c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:59:45.013027 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c624387-b1af-4cf1-8eda-5088c6b608c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:45.013307 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c624387-b1af-4cf1-8eda-5088c6b608c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:45.013580 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c624387-b1af-4cf1-8eda-5088c6b608c6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:45.014230 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c624387-b1af-4cf1-8eda-5088c6b608c6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:59:45.014532 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c624387-b1af-4cf1-8eda-5088c6b608c6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:59:45.016169 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c624387-b1af-4cf1-8eda-5088c6b608c6 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:59:45.025859 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a46084b-bb0a-4db6-a2a8-40d1ab79707e admin admin] RBAC: Authorizing `identity:create_grant(project_id=81ed94c4abf74746bb798c6cf80173ca, user_id=443b625c635944189820425cea6c5722, role_id=de2d2bd26fff4748a79990f97a107164)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:59:45.029138 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a46084b-bb0a-4db6-a2a8-40d1ab79707e admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:59:45.029809 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 618/1244] 10.222.0.59 () {58 vars in 1153 bytes} [Tue Oct 11 11:59:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:59:45.030754 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c624387-b1af-4cf1-8eda-5088c6b608c6 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:59:45.041407 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c624387-b1af-4cf1-8eda-5088c6b608c6 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:59:45.046432 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b37a5f-bf3a-4c78-8894-07855b6f4f6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:59:45.047087 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b37a5f-bf3a-4c78-8894-07855b6f4f6c admin admin] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:45.047367 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b37a5f-bf3a-4c78-8894-07855b6f4f6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:45.047636 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b37a5f-bf3a-4c78-8894-07855b6f4f6c admin admin] PATH_INFO: `/v3/projects` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:45.048271 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b37a5f-bf3a-4c78-8894-07855b6f4f6c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:59:45.049286 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: WARNING py.warnings [None req-92b37a5f-bf3a-4c78-8894-07855b6f4f6c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 11 11:59:45.049286 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: warnings.warn(msg) Oct 11 11:59:45.049286 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: Oct 11 11:59:45.050487 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b37a5f-bf3a-4c78-8894-07855b6f4f6c admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:59:45.051146 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 627/1245] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 11:59:44 2022] PUT /identity/v3/projects/81ed94c4abf74746bb798c6cf80173ca/users/443b625c635944189820425cea6c5722/roles/de2d2bd26fff4748a79990f97a107164 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 11:59:45.052038 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a46084b-bb0a-4db6-a2a8-40d1ab79707e admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:59:45.062059 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:59:45.085292 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 619/1246] 10.222.0.59 () {60 vars in 1153 bytes} [Tue Oct 11 11:59:45 2022] POST /identity/v3/projects => generated 377 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 11:59:45.087508 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b37a5f-bf3a-4c78-8894-07855b6f4f6c admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:59:45.091851 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:59:45.123120 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a46084b-bb0a-4db6-a2a8-40d1ab79707e admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:59:45.152666 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b37a5f-bf3a-4c78-8894-07855b6f4f6c admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:59:45.157884 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7b8e56-adac-4dec-99e5-b518c8e9d5cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:59:45.158503 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7b8e56-adac-4dec-99e5-b518c8e9d5cd admin admin] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:45.158807 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7b8e56-adac-4dec-99e5-b518c8e9d5cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:45.159105 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7b8e56-adac-4dec-99e5-b518c8e9d5cd admin admin] PATH_INFO: `/v3/projects` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:45.159750 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7b8e56-adac-4dec-99e5-b518c8e9d5cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:59:45.161744 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7b8e56-adac-4dec-99e5-b518c8e9d5cd admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:59:45.180716 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634f009f-aa0a-4365-9a4b-4825f9f59cbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:59:45.181310 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634f009f-aa0a-4365-9a4b-4825f9f59cbb admin admin] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:45.181577 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634f009f-aa0a-4365-9a4b-4825f9f59cbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:45.181851 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634f009f-aa0a-4365-9a4b-4825f9f59cbb admin admin] PATH_INFO: `/v3/projects` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:45.182490 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634f009f-aa0a-4365-9a4b-4825f9f59cbb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:59:45.184447 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634f009f-aa0a-4365-9a4b-4825f9f59cbb admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:59:45.190610 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 620/1247] 10.222.0.59 () {60 vars in 1153 bytes} [Tue Oct 11 11:59:45 2022] POST /identity/v3/projects => generated 387 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 11:59:45.191156 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7b8e56-adac-4dec-99e5-b518c8e9d5cd admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:59:45.216024 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 628/1248] 10.222.0.59 () {60 vars in 1153 bytes} [Tue Oct 11 11:59:45 2022] POST /identity/v3/projects => generated 403 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 11:59:45.217021 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634f009f-aa0a-4365-9a4b-4825f9f59cbb admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:59:45.255588 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7b8e56-adac-4dec-99e5-b518c8e9d5cd admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:59:45.279798 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634f009f-aa0a-4365-9a4b-4825f9f59cbb admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:59:45.284448 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaedbcd-5d4d-4281-822e-259a56c270cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:59:45.285060 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaedbcd-5d4d-4281-822e-259a56c270cb admin admin] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:45.285335 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaedbcd-5d4d-4281-822e-259a56c270cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:45.285622 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaedbcd-5d4d-4281-822e-259a56c270cb admin admin] PATH_INFO: `/v3/projects` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:45.286277 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddaedbcd-5d4d-4281-822e-259a56c270cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:59:45.288311 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddaedbcd-5d4d-4281-822e-259a56c270cb admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:59:45.313089 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b27ba8-b87e-4d1e-aeaf-48c9326296f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:59:45.313724 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b27ba8-b87e-4d1e-aeaf-48c9326296f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:45.314025 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b27ba8-b87e-4d1e-aeaf-48c9326296f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:45.314262 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b27ba8-b87e-4d1e-aeaf-48c9326296f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:45.314951 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b27ba8-b87e-4d1e-aeaf-48c9326296f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:59:45.316812 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 629/1249] 10.222.0.59 () {60 vars in 1153 bytes} [Tue Oct 11 11:59:45 2022] POST /identity/v3/projects => generated 385 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 11:59:45.317662 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaedbcd-5d4d-4281-822e-259a56c270cb admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:59:45.318900 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b27ba8-b87e-4d1e-aeaf-48c9326296f6 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:59:45.344933 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:59:45.350090 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 621/1250] 10.222.0.59 () {60 vars in 1153 bytes} [Tue Oct 11 11:59:45 2022] POST /identity/v3/projects => generated 391 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 11:59:45.351098 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b27ba8-b87e-4d1e-aeaf-48c9326296f6 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:59:45.373163 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:59:45.389422 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaedbcd-5d4d-4281-822e-259a56c270cb admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:59:45.418141 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b27ba8-b87e-4d1e-aeaf-48c9326296f6 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:59:45.423190 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217920f5-e5f4-4efb-b468-8818eba94bfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:59:45.423812 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217920f5-e5f4-4efb-b468-8818eba94bfc admin admin] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:45.424098 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217920f5-e5f4-4efb-b468-8818eba94bfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:45.424380 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217920f5-e5f4-4efb-b468-8818eba94bfc admin admin] PATH_INFO: `/v3/projects` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:45.425018 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217920f5-e5f4-4efb-b468-8818eba94bfc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:59:45.427067 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217920f5-e5f4-4efb-b468-8818eba94bfc admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:59:45.448128 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f4c627-20da-4662-9ef7-ba4d6d93c753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:59:45.449015 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f4c627-20da-4662-9ef7-ba4d6d93c753 admin admin] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:45.449511 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f4c627-20da-4662-9ef7-ba4d6d93c753 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:45.449877 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f4c627-20da-4662-9ef7-ba4d6d93c753 admin admin] PATH_INFO: `/v3/projects` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:45.450625 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f4c627-20da-4662-9ef7-ba4d6d93c753 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:59:45.452669 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f4c627-20da-4662-9ef7-ba4d6d93c753 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:59:45.456093 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 622/1251] 10.222.0.59 () {60 vars in 1153 bytes} [Tue Oct 11 11:59:45 2022] POST /identity/v3/projects => generated 385 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 11:59:45.457382 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38bc5b4-cac1-4d76-a724-3ca944cf3d57 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:45.457662 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38bc5b4-cac1-4d76-a724-3ca944cf3d57 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:45.457933 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38bc5b4-cac1-4d76-a724-3ca944cf3d57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:45.484309 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 630/1252] 10.222.0.59 () {60 vars in 1153 bytes} [Tue Oct 11 11:59:45 2022] POST /identity/v3/projects => generated 379 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 11:59:45.484309 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f4c627-20da-4662-9ef7-ba4d6d93c753 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:59:45.487482 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-d38bc5b4-cac1-4d76-a724-3ca944cf3d57 None None] MFA Rules not processed for user `443b625c635944189820425cea6c5722`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:59:45.488718 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-d38bc5b4-cac1-4d76-a724-3ca944cf3d57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:59:45.546778 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f4c627-20da-4662-9ef7-ba4d6d93c753 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:59:45.551760 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9f27de-6c6a-4d87-9d95-c1a96bdd1cae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:59:45.552499 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9f27de-6c6a-4d87-9d95-c1a96bdd1cae admin admin] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:45.552782 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9f27de-6c6a-4d87-9d95-c1a96bdd1cae admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:45.553027 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9f27de-6c6a-4d87-9d95-c1a96bdd1cae admin admin] PATH_INFO: `/v3/users` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:45.553652 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9f27de-6c6a-4d87-9d95-c1a96bdd1cae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:59:45.554710 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: WARNING py.warnings [None req-9e9f27de-6c6a-4d87-9d95-c1a96bdd1cae admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 11 11:59:45.554710 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: warnings.warn(msg) Oct 11 11:59:45.554710 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: Oct 11 11:59:45.555755 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 623/1253] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 11:59:45 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 11:59:45.555910 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9f27de-6c6a-4d87-9d95-c1a96bdd1cae admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:59:45.559528 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38bc5b4-cac1-4d76-a724-3ca944cf3d57 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:59:45.571205 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38bc5b4-cac1-4d76-a724-3ca944cf3d57 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:59:45.580885 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6147af65-d8f2-4614-95c0-dcc026a355be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:59:45.581581 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6147af65-d8f2-4614-95c0-dcc026a355be admin admin] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:45.581865 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6147af65-d8f2-4614-95c0-dcc026a355be admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:45.582138 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6147af65-d8f2-4614-95c0-dcc026a355be admin admin] PATH_INFO: `/v3/users` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:45.582852 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6147af65-d8f2-4614-95c0-dcc026a355be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:59:45.584853 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6147af65-d8f2-4614-95c0-dcc026a355be admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:59:45.586937 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 631/1254] 10.222.0.59 () {60 vars in 1144 bytes} [Tue Oct 11 11:59:45 2022] POST /identity/v3/users => generated 289 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 11:59:45.587917 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9f27de-6c6a-4d87-9d95-c1a96bdd1cae admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:59:45.599833 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9f27de-6c6a-4d87-9d95-c1a96bdd1cae admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:59:45.606152 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01d72de-959a-4a53-87b0-7008e689bbf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:59:45.606867 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01d72de-959a-4a53-87b0-7008e689bbf2 admin admin] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:45.607142 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01d72de-959a-4a53-87b0-7008e689bbf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:45.607420 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01d72de-959a-4a53-87b0-7008e689bbf2 admin admin] PATH_INFO: `/v3/users` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:45.608061 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01d72de-959a-4a53-87b0-7008e689bbf2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:59:45.608736 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 624/1255] 10.222.0.59 () {60 vars in 1144 bytes} [Tue Oct 11 11:59:45 2022] POST /identity/v3/users => generated 294 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 11:59:45.609562 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6147af65-d8f2-4614-95c0-dcc026a355be admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:59:45.610063 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01d72de-959a-4a53-87b0-7008e689bbf2 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:59:45.619569 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6147af65-d8f2-4614-95c0-dcc026a355be admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:59:45.626504 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edb8955-a3ef-42df-b433-5f6da078fad4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:59:45.627301 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edb8955-a3ef-42df-b433-5f6da078fad4 admin admin] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:45.627587 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edb8955-a3ef-42df-b433-5f6da078fad4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:45.627973 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edb8955-a3ef-42df-b433-5f6da078fad4 admin admin] PATH_INFO: `/v3/users` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:45.628518 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edb8955-a3ef-42df-b433-5f6da078fad4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:59:45.630625 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edb8955-a3ef-42df-b433-5f6da078fad4 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:59:45.635017 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 632/1256] 10.222.0.59 () {60 vars in 1144 bytes} [Tue Oct 11 11:59:45 2022] POST /identity/v3/users => generated 302 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 11:59:45.636096 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01d72de-959a-4a53-87b0-7008e689bbf2 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:59:45.646812 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01d72de-959a-4a53-87b0-7008e689bbf2 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:59:45.654143 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2619af0-c547-438f-a2e6-5f0f49f9eae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:59:45.654767 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2619af0-c547-438f-a2e6-5f0f49f9eae4 admin admin] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:45.655043 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2619af0-c547-438f-a2e6-5f0f49f9eae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:45.655309 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2619af0-c547-438f-a2e6-5f0f49f9eae4 admin admin] PATH_INFO: `/v3/users` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:45.655953 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2619af0-c547-438f-a2e6-5f0f49f9eae4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:59:45.657864 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2619af0-c547-438f-a2e6-5f0f49f9eae4 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:59:45.664200 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 625/1257] 10.222.0.59 () {60 vars in 1144 bytes} [Tue Oct 11 11:59:45 2022] POST /identity/v3/users => generated 293 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 11:59:45.664200 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edb8955-a3ef-42df-b433-5f6da078fad4 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:59:45.672682 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edb8955-a3ef-42df-b433-5f6da078fad4 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:59:45.678148 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4711d9-6911-41ab-b3a2-70819359514a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:59:45.678794 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4711d9-6911-41ab-b3a2-70819359514a admin admin] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:45.679073 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4711d9-6911-41ab-b3a2-70819359514a admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:45.679377 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4711d9-6911-41ab-b3a2-70819359514a admin admin] PATH_INFO: `/v3/users` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:45.680018 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4711d9-6911-41ab-b3a2-70819359514a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:59:45.681151 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 633/1258] 10.222.0.59 () {60 vars in 1144 bytes} [Tue Oct 11 11:59:45 2022] POST /identity/v3/users => generated 296 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 11:59:45.681983 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2619af0-c547-438f-a2e6-5f0f49f9eae4 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:59:45.683531 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4711d9-6911-41ab-b3a2-70819359514a admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:59:45.691763 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2619af0-c547-438f-a2e6-5f0f49f9eae4 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:59:45.698145 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b3636b-4436-471e-877d-4f50310ed04e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:59:45.699043 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b3636b-4436-471e-877d-4f50310ed04e admin admin] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:45.699513 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b3636b-4436-471e-877d-4f50310ed04e admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:45.700033 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b3636b-4436-471e-877d-4f50310ed04e admin admin] PATH_INFO: `/v3/users` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:45.700654 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b3636b-4436-471e-877d-4f50310ed04e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:59:45.702623 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b3636b-4436-471e-877d-4f50310ed04e admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:59:45.710356 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 626/1259] 10.222.0.59 () {60 vars in 1144 bytes} [Tue Oct 11 11:59:45 2022] POST /identity/v3/users => generated 293 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 11:59:45.712126 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=89,keystone:INSERT=11 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:59:45.712526 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276689e3-8574-4eb1-bbbb-caba60253446 None None] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:45.712815 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276689e3-8574-4eb1-bbbb-caba60253446 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:45.713621 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276689e3-8574-4eb1-bbbb-caba60253446 None None] PATH_INFO: `/` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:45.715571 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 627/1260] 10.222.0.59 () {54 vars in 961 bytes} [Tue Oct 11 11:59:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:59:45.716580 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276689e3-8574-4eb1-bbbb-caba60253446 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:59:45.732395 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 634/1261] 10.222.0.59 () {60 vars in 1144 bytes} [Tue Oct 11 11:59:45 2022] POST /identity/v3/users => generated 290 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 11:59:45.732395 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b3636b-4436-471e-877d-4f50310ed04e admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:59:45.732730 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276689e3-8574-4eb1-bbbb-caba60253446 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:59:45.736673 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277adb01-3705-4279-9499-e234c56680bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:59:45.737379 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277adb01-3705-4279-9499-e234c56680bf admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:45.737824 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277adb01-3705-4279-9499-e234c56680bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:45.738116 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277adb01-3705-4279-9499-e234c56680bf admin admin] PATH_INFO: `/v3/roles` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:45.738855 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277adb01-3705-4279-9499-e234c56680bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:59:45.740614 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277adb01-3705-4279-9499-e234c56680bf admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:59:45.748026 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b3636b-4436-471e-877d-4f50310ed04e admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:59:45.752624 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 628/1262] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 11:59:45 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 11:59:45.752882 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e356d4c-529d-4079-bb4d-8a6a245786aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:59:45.753479 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e356d4c-529d-4079-bb4d-8a6a245786aa admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:45.753608 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277adb01-3705-4279-9499-e234c56680bf admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:59:45.753807 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e356d4c-529d-4079-bb4d-8a6a245786aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:45.754022 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e356d4c-529d-4079-bb4d-8a6a245786aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:45.754724 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e356d4c-529d-4079-bb4d-8a6a245786aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:59:45.756359 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e356d4c-529d-4079-bb4d-8a6a245786aa admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:59:45.766444 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277adb01-3705-4279-9499-e234c56680bf admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:59:45.767656 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 635/1263] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 11:59:45 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 11:59:45.768511 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e356d4c-529d-4079-bb4d-8a6a245786aa admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:59:45.774858 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887de167-3c33-4b00-9801-57ea6651f214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:59:45.775487 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887de167-3c33-4b00-9801-57ea6651f214 admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:45.775775 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887de167-3c33-4b00-9801-57ea6651f214 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:45.776040 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887de167-3c33-4b00-9801-57ea6651f214 admin admin] PATH_INFO: `/v3/roles` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:45.776695 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887de167-3c33-4b00-9801-57ea6651f214 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:59:45.778690 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887de167-3c33-4b00-9801-57ea6651f214 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:59:45.779709 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e356d4c-529d-4079-bb4d-8a6a245786aa admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:59:45.785138 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d5937f-5a2c-4c22-b0bd-6f74913b2c77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:59:45.785752 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d5937f-5a2c-4c22-b0bd-6f74913b2c77 admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:45.786029 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d5937f-5a2c-4c22-b0bd-6f74913b2c77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:45.786301 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d5937f-5a2c-4c22-b0bd-6f74913b2c77 admin admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:45.786986 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d5937f-5a2c-4c22-b0bd-6f74913b2c77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:59:45.788594 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d5937f-5a2c-4c22-b0bd-6f74913b2c77 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:59:45.792509 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 629/1264] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 11:59:45 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 11:59:45.792509 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887de167-3c33-4b00-9801-57ea6651f214 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:59:45.800347 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 636/1265] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 11:59:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 11:59:45.800673 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887de167-3c33-4b00-9801-57ea6651f214 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:59:45.801131 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d5937f-5a2c-4c22-b0bd-6f74913b2c77 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:59:45.806643 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b77be5-55e8-4054-8619-a2b202d09cb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:59:45.807292 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b77be5-55e8-4054-8619-a2b202d09cb3 admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:45.807583 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b77be5-55e8-4054-8619-a2b202d09cb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:45.807893 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b77be5-55e8-4054-8619-a2b202d09cb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:45.808514 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b77be5-55e8-4054-8619-a2b202d09cb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:59:45.810146 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b77be5-55e8-4054-8619-a2b202d09cb3 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:59:45.812127 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d5937f-5a2c-4c22-b0bd-6f74913b2c77 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:59:45.817016 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d5eb6d-b9c9-4aad-9cff-cb3db19a659e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:59:45.817635 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d5eb6d-b9c9-4aad-9cff-cb3db19a659e admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:45.817911 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d5eb6d-b9c9-4aad-9cff-cb3db19a659e admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:45.818183 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d5eb6d-b9c9-4aad-9cff-cb3db19a659e admin admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:45.818867 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d5eb6d-b9c9-4aad-9cff-cb3db19a659e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:59:45.820476 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d5eb6d-b9c9-4aad-9cff-cb3db19a659e admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:59:45.821385 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 630/1266] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 11:59:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 11:59:45.823714 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ec7f74-77e5-413b-8cf8-f5c0f152d334 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:45.824006 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ec7f74-77e5-413b-8cf8-f5c0f152d334 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:45.824559 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ec7f74-77e5-413b-8cf8-f5c0f152d334 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:45.831520 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 637/1267] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 11:59:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 11:59:45.833339 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d5eb6d-b9c9-4aad-9cff-cb3db19a659e admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:59:45.845048 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:59:45.854321 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d5eb6d-b9c9-4aad-9cff-cb3db19a659e admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:59:45.859388 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2122bd-2063-4240-8b91-e86d9bad3536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:59:45.860028 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2122bd-2063-4240-8b91-e86d9bad3536 admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:45.860293 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2122bd-2063-4240-8b91-e86d9bad3536 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:45.860565 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2122bd-2063-4240-8b91-e86d9bad3536 admin admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:45.861219 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2122bd-2063-4240-8b91-e86d9bad3536 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:59:45.862875 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2122bd-2063-4240-8b91-e86d9bad3536 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:59:45.872651 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-b3ec7f74-77e5-413b-8cf8-f5c0f152d334 None None] MFA Rules not processed for user `bad5e1cb306b46818096940acc3810b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:59:45.873877 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-b3ec7f74-77e5-413b-8cf8-f5c0f152d334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:59:45.874430 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 638/1268] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 11:59:45 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 11:59:45.875146 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2122bd-2063-4240-8b91-e86d9bad3536 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:59:45.886652 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2122bd-2063-4240-8b91-e86d9bad3536 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:59:45.891321 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87090681-2278-4b8a-8ca8-f4740ba12696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:59:45.891923 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87090681-2278-4b8a-8ca8-f4740ba12696 admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:45.892207 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87090681-2278-4b8a-8ca8-f4740ba12696 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:45.892481 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87090681-2278-4b8a-8ca8-f4740ba12696 admin admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:45.893127 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87090681-2278-4b8a-8ca8-f4740ba12696 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:59:45.894769 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87090681-2278-4b8a-8ca8-f4740ba12696 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:59:45.905845 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 639/1269] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 11:59:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 11:59:45.906792 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87090681-2278-4b8a-8ca8-f4740ba12696 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:59:45.917484 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87090681-2278-4b8a-8ca8-f4740ba12696 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:59:45.923102 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b12a6b-b12d-460a-ae2f-305c21b4b935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:59:45.923732 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b12a6b-b12d-460a-ae2f-305c21b4b935 admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:45.924014 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b12a6b-b12d-460a-ae2f-305c21b4b935 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:45.924289 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b12a6b-b12d-460a-ae2f-305c21b4b935 admin admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:45.924968 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b12a6b-b12d-460a-ae2f-305c21b4b935 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:59:45.926775 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b12a6b-b12d-460a-ae2f-305c21b4b935 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:59:45.938225 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 640/1270] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 11:59:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 11:59:45.939149 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b12a6b-b12d-460a-ae2f-305c21b4b935 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:59:45.949244 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b12a6b-b12d-460a-ae2f-305c21b4b935 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:59:45.952141 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 631/1271] 10.222.0.59 () {58 vars in 1058 bytes} [Tue Oct 11 11:59:45 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 11:59:45.952831 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ec7f74-77e5-413b-8cf8-f5c0f152d334 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:59:45.954584 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efaf770-6631-4f27-8ad3-88f5dfe5a2da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:59:45.955216 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efaf770-6631-4f27-8ad3-88f5dfe5a2da admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:45.955500 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efaf770-6631-4f27-8ad3-88f5dfe5a2da admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:45.955764 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efaf770-6631-4f27-8ad3-88f5dfe5a2da admin admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:45.956407 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efaf770-6631-4f27-8ad3-88f5dfe5a2da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:59:45.958079 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efaf770-6631-4f27-8ad3-88f5dfe5a2da admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:59:45.964265 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ec7f74-77e5-413b-8cf8-f5c0f152d334 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:59:45.969289 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5eccb0d-2c3d-411e-8fb9-32138718a8d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:59:45.969918 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eccb0d-2c3d-411e-8fb9-32138718a8d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:45.970288 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eccb0d-2c3d-411e-8fb9-32138718a8d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:45.970585 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eccb0d-2c3d-411e-8fb9-32138718a8d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:45.971285 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5eccb0d-2c3d-411e-8fb9-32138718a8d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:59:45.972893 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5eccb0d-2c3d-411e-8fb9-32138718a8d2 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:59:45.973636 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 641/1272] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 11:59:45 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 11:59:45.974924 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efaf770-6631-4f27-8ad3-88f5dfe5a2da admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:59:45.983847 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 632/1273] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 11:59:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 11:59:45.984745 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5eccb0d-2c3d-411e-8fb9-32138718a8d2 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:59:45.986221 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efaf770-6631-4f27-8ad3-88f5dfe5a2da admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:59:45.991373 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fa8a20-ff04-4aa6-afaa-fcd43621918a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:59:45.992052 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fa8a20-ff04-4aa6-afaa-fcd43621918a admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:45.992354 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fa8a20-ff04-4aa6-afaa-fcd43621918a admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:45.992648 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fa8a20-ff04-4aa6-afaa-fcd43621918a admin admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:45.993308 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fa8a20-ff04-4aa6-afaa-fcd43621918a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:59:45.995013 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fa8a20-ff04-4aa6-afaa-fcd43621918a admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:59:45.995153 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5eccb0d-2c3d-411e-8fb9-32138718a8d2 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:59:46.001473 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d5fa97-6dd5-4018-bedb-63d4d176539b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:59:46.001473 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d5fa97-6dd5-4018-bedb-63d4d176539b admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:46.002046 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d5fa97-6dd5-4018-bedb-63d4d176539b admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:46.002214 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d5fa97-6dd5-4018-bedb-63d4d176539b admin admin] PATH_INFO: `/v3/roles` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:46.003060 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d5fa97-6dd5-4018-bedb-63d4d176539b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:59:46.004746 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d5fa97-6dd5-4018-bedb-63d4d176539b admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:59:46.005786 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 642/1274] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 11:59:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 11:59:46.006634 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fa8a20-ff04-4aa6-afaa-fcd43621918a admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:59:46.019353 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 633/1275] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 11:59:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 11:59:46.019553 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fa8a20-ff04-4aa6-afaa-fcd43621918a admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:59:46.019658 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d5fa97-6dd5-4018-bedb-63d4d176539b admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:59:46.021921 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dd4985-7dbb-408b-a64d-1d97aac57dc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:59:46.022527 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dd4985-7dbb-408b-a64d-1d97aac57dc4 admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:46.022838 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dd4985-7dbb-408b-a64d-1d97aac57dc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:46.023134 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dd4985-7dbb-408b-a64d-1d97aac57dc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:46.023858 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4dd4985-7dbb-408b-a64d-1d97aac57dc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:59:46.025474 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4dd4985-7dbb-408b-a64d-1d97aac57dc4 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:59:46.029414 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d5fa97-6dd5-4018-bedb-63d4d176539b admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:59:46.034196 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f0ced4-1caf-4067-8b85-1965749855c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:59:46.034889 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f0ced4-1caf-4067-8b85-1965749855c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:46.035167 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f0ced4-1caf-4067-8b85-1965749855c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:46.035437 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f0ced4-1caf-4067-8b85-1965749855c9 admin admin] PATH_INFO: `/v3/projects/2e24b09232504f99be424f15212efeb2/users/a1cbbbbde3264f51b0038a7708056c6d/roles/de2d2bd26fff4748a79990f97a107164` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:46.037138 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 643/1276] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 11:59:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 11:59:46.040205 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dd4985-7dbb-408b-a64d-1d97aac57dc4 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:59:46.052082 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f0ced4-1caf-4067-8b85-1965749855c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e24b09232504f99be424f15212efeb2, user_id=a1cbbbbde3264f51b0038a7708056c6d, role_id=de2d2bd26fff4748a79990f97a107164)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:59:46.052266 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dd4985-7dbb-408b-a64d-1d97aac57dc4 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:59:46.054734 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f0ced4-1caf-4067-8b85-1965749855c9 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:59:46.056778 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17a648b-3b2b-4751-8ff1-d537ebe9fb93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:59:46.057448 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17a648b-3b2b-4751-8ff1-d537ebe9fb93 admin admin] REQUEST_METHOD: `PUT` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:46.057726 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17a648b-3b2b-4751-8ff1-d537ebe9fb93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:46.058005 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17a648b-3b2b-4751-8ff1-d537ebe9fb93 admin admin] PATH_INFO: `/v3/projects/0b77d55b7953427493baf1ed9f855c83/users/37e06773cb10450cb28d42d98f7c9f21/roles/de2d2bd26fff4748a79990f97a107164` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:46.068988 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 634/1277] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 11:59:46 2022] PUT /identity/v3/projects/2e24b09232504f99be424f15212efeb2/users/a1cbbbbde3264f51b0038a7708056c6d/roles/de2d2bd26fff4748a79990f97a107164 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 11:59:46.069910 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f0ced4-1caf-4067-8b85-1965749855c9 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:59:46.085189 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17a648b-3b2b-4751-8ff1-d537ebe9fb93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b77d55b7953427493baf1ed9f855c83, user_id=37e06773cb10450cb28d42d98f7c9f21, role_id=de2d2bd26fff4748a79990f97a107164)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:59:46.088002 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17a648b-3b2b-4751-8ff1-d537ebe9fb93 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:59:46.103725 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 644/1278] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 11:59:46 2022] PUT /identity/v3/projects/0b77d55b7953427493baf1ed9f855c83/users/37e06773cb10450cb28d42d98f7c9f21/roles/de2d2bd26fff4748a79990f97a107164 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 11:59:46.105851 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17a648b-3b2b-4751-8ff1-d537ebe9fb93 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:59:46.147130 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f0ced4-1caf-4067-8b85-1965749855c9 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:59:46.182593 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17a648b-3b2b-4751-8ff1-d537ebe9fb93 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:59:46.187877 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7dc96d-4f47-4b4f-9fc3-2bed8ebc585d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:59:46.188584 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7dc96d-4f47-4b4f-9fc3-2bed8ebc585d admin admin] REQUEST_METHOD: `PUT` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:46.188866 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7dc96d-4f47-4b4f-9fc3-2bed8ebc585d admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:46.189145 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7dc96d-4f47-4b4f-9fc3-2bed8ebc585d admin admin] PATH_INFO: `/v3/projects/c2e9fa99631f4b0b9c3cfe2cf33a8a82/users/449cae8c746b47a9bbedec9a845cb9bc/roles/de2d2bd26fff4748a79990f97a107164` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:46.191606 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:59:46.204513 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7dc96d-4f47-4b4f-9fc3-2bed8ebc585d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2e9fa99631f4b0b9c3cfe2cf33a8a82, user_id=449cae8c746b47a9bbedec9a845cb9bc, role_id=de2d2bd26fff4748a79990f97a107164)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:59:46.207234 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7dc96d-4f47-4b4f-9fc3-2bed8ebc585d admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:59:46.219840 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30a5a29-ce92-485e-a453-da53a4b9db2a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bad5e1cb306b46818096940acc3810b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:59:46.220462 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30a5a29-ce92-485e-a453-da53a4b9db2a service nova] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:46.220772 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30a5a29-ce92-485e-a453-da53a4b9db2a service nova] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:46.220915 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 645/1279] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 11:59:46 2022] PUT /identity/v3/projects/c2e9fa99631f4b0b9c3cfe2cf33a8a82/users/449cae8c746b47a9bbedec9a845cb9bc/roles/de2d2bd26fff4748a79990f97a107164 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 11:59:46.221013 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30a5a29-ce92-485e-a453-da53a4b9db2a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:46.221792 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7dc96d-4f47-4b4f-9fc3-2bed8ebc585d admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:59:46.239612 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:59:46.281364 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30a5a29-ce92-485e-a453-da53a4b9db2a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:59:46.283084 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30a5a29-ce92-485e-a453-da53a4b9db2a service nova] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:59:46.291163 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7dc96d-4f47-4b4f-9fc3-2bed8ebc585d admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:59:46.294329 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 635/1280] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 11:59:46 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 11:59:46.295240 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30a5a29-ce92-485e-a453-da53a4b9db2a service nova] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:59:46.296304 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a172c9-5085-409a-bd70-2f9442ed59e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:59:46.296979 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a172c9-5085-409a-bd70-2f9442ed59e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:46.297243 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a172c9-5085-409a-bd70-2f9442ed59e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:46.297518 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a172c9-5085-409a-bd70-2f9442ed59e7 admin admin] PATH_INFO: `/v3/projects/a5e9e0788b5c42c9837666a2403c87cc/users/9c3e056a705e4d19972f77f507c7f95c/roles/de2d2bd26fff4748a79990f97a107164` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:46.307688 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30a5a29-ce92-485e-a453-da53a4b9db2a service nova] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:59:46.312311 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9304633-703e-4cff-b661-5b14e7b17e42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:59:46.313028 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9304633-703e-4cff-b661-5b14e7b17e42 admin admin] REQUEST_METHOD: `PUT` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:46.313312 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9304633-703e-4cff-b661-5b14e7b17e42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:46.313580 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9304633-703e-4cff-b661-5b14e7b17e42 admin admin] PATH_INFO: `/v3/projects/e3e6a648f46646cea42b910d02e0628f/users/c58107eee1b94541a330c26d7b82a53b/roles/de2d2bd26fff4748a79990f97a107164` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:46.314523 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a172c9-5085-409a-bd70-2f9442ed59e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5e9e0788b5c42c9837666a2403c87cc, user_id=9c3e056a705e4d19972f77f507c7f95c, role_id=de2d2bd26fff4748a79990f97a107164)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:59:46.317646 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a172c9-5085-409a-bd70-2f9442ed59e7 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:59:46.331104 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9304633-703e-4cff-b661-5b14e7b17e42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3e6a648f46646cea42b910d02e0628f, user_id=c58107eee1b94541a330c26d7b82a53b, role_id=de2d2bd26fff4748a79990f97a107164)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:59:46.333820 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9304633-703e-4cff-b661-5b14e7b17e42 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:59:46.334482 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 646/1281] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 11:59:46 2022] PUT /identity/v3/projects/a5e9e0788b5c42c9837666a2403c87cc/users/9c3e056a705e4d19972f77f507c7f95c/roles/de2d2bd26fff4748a79990f97a107164 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 11:59:46.335386 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a172c9-5085-409a-bd70-2f9442ed59e7 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:59:46.359702 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 636/1282] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 11:59:46 2022] PUT /identity/v3/projects/e3e6a648f46646cea42b910d02e0628f/users/c58107eee1b94541a330c26d7b82a53b/roles/de2d2bd26fff4748a79990f97a107164 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 11:59:46.360396 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9304633-703e-4cff-b661-5b14e7b17e42 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:59:46.412275 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a172c9-5085-409a-bd70-2f9442ed59e7 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:59:46.433271 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9304633-703e-4cff-b661-5b14e7b17e42 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:59:46.438589 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf80ed4-e877-4bac-b361-b31033e3e916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:59:46.439131 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf80ed4-e877-4bac-b361-b31033e3e916 admin admin] REQUEST_METHOD: `PUT` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:46.439413 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf80ed4-e877-4bac-b361-b31033e3e916 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:46.439834 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf80ed4-e877-4bac-b361-b31033e3e916 admin admin] PATH_INFO: `/v3/projects/bbb66bfd69114f489c0bced30cc1b0b9/users/d1619ed83ae546a7ac2219c055ca4076/roles/de2d2bd26fff4748a79990f97a107164` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:46.455936 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf80ed4-e877-4bac-b361-b31033e3e916 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bbb66bfd69114f489c0bced30cc1b0b9, user_id=d1619ed83ae546a7ac2219c055ca4076, role_id=de2d2bd26fff4748a79990f97a107164)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:59:46.458449 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf80ed4-e877-4bac-b361-b31033e3e916 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:59:46.471301 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd5929c-3137-4bc7-824a-87fd83bb25fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:59:46.471974 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd5929c-3137-4bc7-824a-87fd83bb25fc admin admin] REQUEST_METHOD: `PUT` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:46.472248 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd5929c-3137-4bc7-824a-87fd83bb25fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:46.472523 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd5929c-3137-4bc7-824a-87fd83bb25fc admin admin] PATH_INFO: `/v3/projects/c83f971a0ae04288a5c6dc87d3370979/users/41d3680dda7c4cc8aeb6d086efca0775/roles/de2d2bd26fff4748a79990f97a107164` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:46.473620 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 637/1283] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 11:59:46 2022] PUT /identity/v3/projects/bbb66bfd69114f489c0bced30cc1b0b9/users/d1619ed83ae546a7ac2219c055ca4076/roles/de2d2bd26fff4748a79990f97a107164 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 11:59:46.475252 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59f902b-34aa-4789-a5d7-3f21cfddffb2 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:46.475545 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59f902b-34aa-4789-a5d7-3f21cfddffb2 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:46.475825 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59f902b-34aa-4789-a5d7-3f21cfddffb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:46.491766 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd5929c-3137-4bc7-824a-87fd83bb25fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=c83f971a0ae04288a5c6dc87d3370979, user_id=41d3680dda7c4cc8aeb6d086efca0775, role_id=de2d2bd26fff4748a79990f97a107164)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:59:46.494483 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd5929c-3137-4bc7-824a-87fd83bb25fc admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:59:46.505727 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-f59f902b-34aa-4789-a5d7-3f21cfddffb2 None None] MFA Rules not processed for user `a1cbbbbde3264f51b0038a7708056c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:59:46.506868 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-f59f902b-34aa-4789-a5d7-3f21cfddffb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:59:46.514143 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 647/1284] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 11:59:46 2022] PUT /identity/v3/projects/c83f971a0ae04288a5c6dc87d3370979/users/41d3680dda7c4cc8aeb6d086efca0775/roles/de2d2bd26fff4748a79990f97a107164 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 11:59:46.515376 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fec735f-7c4f-41e6-aef0-3f1b01012881 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:46.515664 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fec735f-7c4f-41e6-aef0-3f1b01012881 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:46.515941 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fec735f-7c4f-41e6-aef0-3f1b01012881 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:46.538650 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:59:46.547192 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-6fec735f-7c4f-41e6-aef0-3f1b01012881 None None] MFA Rules not processed for user `37e06773cb10450cb28d42d98f7c9f21`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:59:46.548746 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-6fec735f-7c4f-41e6-aef0-3f1b01012881 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:59:46.562117 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:59:46.585850 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 638/1285] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 11:59:46 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 11:59:46.590995 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a0bb66-2652-4881-a999-4984d8ae39ea None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:46.591286 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a0bb66-2652-4881-a999-4984d8ae39ea None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:46.591554 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a0bb66-2652-4881-a999-4984d8ae39ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:46.620449 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-36a0bb66-2652-4881-a999-4984d8ae39ea None None] MFA Rules not processed for user `449cae8c746b47a9bbedec9a845cb9bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:59:46.621885 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-36a0bb66-2652-4881-a999-4984d8ae39ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:59:46.622022 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 648/1286] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 11:59:46 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 11:59:46.623018 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4665f8ec-7f01-43a4-803c-a2833e99d245 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:46.623301 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4665f8ec-7f01-43a4-803c-a2833e99d245 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:46.623578 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4665f8ec-7f01-43a4-803c-a2833e99d245 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:46.653050 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-4665f8ec-7f01-43a4-803c-a2833e99d245 None None] MFA Rules not processed for user `9c3e056a705e4d19972f77f507c7f95c`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:59:46.655019 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-4665f8ec-7f01-43a4-803c-a2833e99d245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:59:46.691030 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 639/1287] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 11:59:46 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 11:59:46.692448 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abdd7ab-13e9-48c3-83dc-421b9fa977dd None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:46.692643 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abdd7ab-13e9-48c3-83dc-421b9fa977dd None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:46.692930 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abdd7ab-13e9-48c3-83dc-421b9fa977dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:46.712592 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-4abdd7ab-13e9-48c3-83dc-421b9fa977dd None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:59:46.713820 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-4abdd7ab-13e9-48c3-83dc-421b9fa977dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:59:46.727120 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 649/1288] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 11:59:46 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 11:59:46.728809 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02391b0e-dd49-49d1-920a-d8ce7b6dea52 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:46.729097 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02391b0e-dd49-49d1-920a-d8ce7b6dea52 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:46.729369 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02391b0e-dd49-49d1-920a-d8ce7b6dea52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:46.748608 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-02391b0e-dd49-49d1-920a-d8ce7b6dea52 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:59:46.749808 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-02391b0e-dd49-49d1-920a-d8ce7b6dea52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:59:46.776878 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 640/1289] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 11:59:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 11:59:46.778895 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8806a7-7ace-4b87-a29d-59b11b0a1b05 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:46.779171 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8806a7-7ace-4b87-a29d-59b11b0a1b05 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:46.779448 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8806a7-7ace-4b87-a29d-59b11b0a1b05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:46.799233 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-bf8806a7-7ace-4b87-a29d-59b11b0a1b05 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:59:46.800478 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-bf8806a7-7ace-4b87-a29d-59b11b0a1b05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:59:46.808359 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 641/1290] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 11:59:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 11:59:46.809606 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e21d36-d90d-4527-bc86-989b54dab2c2 None None] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:46.809902 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e21d36-d90d-4527-bc86-989b54dab2c2 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:46.810172 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e21d36-d90d-4527-bc86-989b54dab2c2 None None] PATH_INFO: `/` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:46.811605 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 642/1291] 10.222.0.59 () {54 vars in 912 bytes} [Tue Oct 11 11:59:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:59:46.812185 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e21d36-d90d-4527-bc86-989b54dab2c2 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:59:46.820538 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 650/1292] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 11:59:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 11:59:46.820538 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02391b0e-dd49-49d1-920a-d8ce7b6dea52 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:59:46.827755 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e21d36-d90d-4527-bc86-989b54dab2c2 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:59:46.832138 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02391b0e-dd49-49d1-920a-d8ce7b6dea52 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:59:46.832396 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec373a8-67dc-46f0-ae59-e64e010949cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:59:46.833145 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec373a8-67dc-46f0-ae59-e64e010949cb admin admin] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:46.833428 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec373a8-67dc-46f0-ae59-e64e010949cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:46.833674 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec373a8-67dc-46f0-ae59-e64e010949cb admin admin] PATH_INFO: `/v3/projects` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:46.834317 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec373a8-67dc-46f0-ae59-e64e010949cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:59:46.836313 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec373a8-67dc-46f0-ae59-e64e010949cb admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:59:46.837476 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30fd05c-1a65-47b9-94fa-df488a206d2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:59:46.838221 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30fd05c-1a65-47b9-94fa-df488a206d2f admin admin] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:46.838399 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30fd05c-1a65-47b9-94fa-df488a206d2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:46.838822 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30fd05c-1a65-47b9-94fa-df488a206d2f admin admin] PATH_INFO: `/v3/projects` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:46.839474 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30fd05c-1a65-47b9-94fa-df488a206d2f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:59:46.841360 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30fd05c-1a65-47b9-94fa-df488a206d2f admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:59:46.871182 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 643/1293] 10.222.0.59 () {60 vars in 1153 bytes} [Tue Oct 11 11:59:46 2022] POST /identity/v3/projects => generated 375 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 11:59:46.871182 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec373a8-67dc-46f0-ae59-e64e010949cb admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:59:46.872754 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 651/1294] 10.222.0.59 () {60 vars in 1153 bytes} [Tue Oct 11 11:59:46 2022] POST /identity/v3/projects => generated 385 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 11:59:46.873827 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30fd05c-1a65-47b9-94fa-df488a206d2f admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:59:46.900296 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:59:46.905787 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:59:46.941654 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec373a8-67dc-46f0-ae59-e64e010949cb admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:59:46.946203 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30fd05c-1a65-47b9-94fa-df488a206d2f admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:59:46.946444 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9561658-f4c0-4de3-a03d-5467f6df86ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:59:46.947114 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9561658-f4c0-4de3-a03d-5467f6df86ea admin admin] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:46.947397 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9561658-f4c0-4de3-a03d-5467f6df86ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:46.947671 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9561658-f4c0-4de3-a03d-5467f6df86ea admin admin] PATH_INFO: `/v3/projects` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:46.948319 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9561658-f4c0-4de3-a03d-5467f6df86ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:59:46.950289 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9561658-f4c0-4de3-a03d-5467f6df86ea admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:59:46.951215 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d72e0b4-583e-4dc5-8085-160486a2b074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:59:46.951882 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d72e0b4-583e-4dc5-8085-160486a2b074 admin admin] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:46.952160 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d72e0b4-583e-4dc5-8085-160486a2b074 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:46.952430 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d72e0b4-583e-4dc5-8085-160486a2b074 admin admin] PATH_INFO: `/v3/projects` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:46.953081 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d72e0b4-583e-4dc5-8085-160486a2b074 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:59:46.955093 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d72e0b4-583e-4dc5-8085-160486a2b074 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:59:46.984597 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 644/1295] 10.222.0.59 () {60 vars in 1153 bytes} [Tue Oct 11 11:59:46 2022] POST /identity/v3/projects => generated 405 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 11:59:46.985502 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9561658-f4c0-4de3-a03d-5467f6df86ea admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:59:46.987009 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 652/1296] 10.222.0.59 () {60 vars in 1153 bytes} [Tue Oct 11 11:59:46 2022] POST /identity/v3/projects => generated 385 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 11:59:46.988527 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f82109-aace-4ac6-803a-c50357269087 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:46.988819 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f82109-aace-4ac6-803a-c50357269087 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:46.989140 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f82109-aace-4ac6-803a-c50357269087 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:47.009124 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-a2f82109-aace-4ac6-803a-c50357269087 None None] MFA Rules not processed for user `bad5e1cb306b46818096940acc3810b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:59:47.010350 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-a2f82109-aace-4ac6-803a-c50357269087 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:59:47.045549 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9561658-f4c0-4de3-a03d-5467f6df86ea admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:59:47.077093 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 653/1297] 10.222.0.59 () {58 vars in 1009 bytes} [Tue Oct 11 11:59:46 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 11:59:47.078026 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f82109-aace-4ac6-803a-c50357269087 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:59:47.101156 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0dcaad-a1a0-4eda-9602-58f970222b14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e563fe174db4cc19ae6bf8162cc3d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:59:47.101775 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0dcaad-a1a0-4eda-9602-58f970222b14 service neutron] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:47.102059 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0dcaad-a1a0-4eda-9602-58f970222b14 service neutron] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:47.102330 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0dcaad-a1a0-4eda-9602-58f970222b14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:47.134532 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f82109-aace-4ac6-803a-c50357269087 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:59:47.138247 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6858a58-0448-466e-98b9-fb578b37139b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e563fe174db4cc19ae6bf8162cc3d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:59:47.138868 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6858a58-0448-466e-98b9-fb578b37139b service neutron] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:47.139153 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6858a58-0448-466e-98b9-fb578b37139b service neutron] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:47.139439 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6858a58-0448-466e-98b9-fb578b37139b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:47.166663 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0dcaad-a1a0-4eda-9602-58f970222b14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:59:47.168769 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0dcaad-a1a0-4eda-9602-58f970222b14 service neutron] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:59:47.178705 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 645/1298] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 11:59:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 11:59:47.179669 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0dcaad-a1a0-4eda-9602-58f970222b14 service neutron] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:59:47.181078 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-ba0dcaad-a1a0-4eda-9602-58f970222b14 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:59:47.193886 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0dcaad-a1a0-4eda-9602-58f970222b14 service neutron] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:59:47.198183 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:59:47.198952 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0315c3ba-d62d-4f1f-9e3d-84989bf60ab2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e563fe174db4cc19ae6bf8162cc3d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:59:47.200945 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0315c3ba-d62d-4f1f-9e3d-84989bf60ab2 service neutron] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:47.201364 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0315c3ba-d62d-4f1f-9e3d-84989bf60ab2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:47.201586 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0315c3ba-d62d-4f1f-9e3d-84989bf60ab2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:47.211704 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6858a58-0448-466e-98b9-fb578b37139b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:59:47.214352 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6858a58-0448-466e-98b9-fb578b37139b service neutron] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:59:47.215506 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0315c3ba-d62d-4f1f-9e3d-84989bf60ab2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:59:47.217480 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0315c3ba-d62d-4f1f-9e3d-84989bf60ab2 service neutron] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:59:47.225131 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 654/1299] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 11:59:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 11:59:47.226068 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6858a58-0448-466e-98b9-fb578b37139b service neutron] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:59:47.230072 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 646/1300] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 11:59:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 11:59:47.231372 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0315c3ba-d62d-4f1f-9e3d-84989bf60ab2 service neutron] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:59:47.236744 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6858a58-0448-466e-98b9-fb578b37139b service neutron] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:59:47.239231 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:59:47.243571 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c160f3-15f2-419b-8c40-3f2a28d45c6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:59:47.244286 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0315c3ba-d62d-4f1f-9e3d-84989bf60ab2 service neutron] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:59:47.244435 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c160f3-15f2-419b-8c40-3f2a28d45c6c admin admin] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:47.244581 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c160f3-15f2-419b-8c40-3f2a28d45c6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:47.245553 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c160f3-15f2-419b-8c40-3f2a28d45c6c admin admin] PATH_INFO: `/v3/users` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:47.246262 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c160f3-15f2-419b-8c40-3f2a28d45c6c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:59:47.248655 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c160f3-15f2-419b-8c40-3f2a28d45c6c admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:59:47.255218 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe7bc74-1300-4a9e-8cc9-202bf6ed96ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:59:47.256275 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe7bc74-1300-4a9e-8cc9-202bf6ed96ff admin admin] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:47.256759 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe7bc74-1300-4a9e-8cc9-202bf6ed96ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:47.257215 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe7bc74-1300-4a9e-8cc9-202bf6ed96ff admin admin] PATH_INFO: `/v3/users` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:47.258207 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe7bc74-1300-4a9e-8cc9-202bf6ed96ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:59:47.262421 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe7bc74-1300-4a9e-8cc9-202bf6ed96ff admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:59:47.280923 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 655/1301] 10.222.0.59 () {60 vars in 1144 bytes} [Tue Oct 11 11:59:47 2022] POST /identity/v3/users => generated 294 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 11:59:47.281936 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c160f3-15f2-419b-8c40-3f2a28d45c6c admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:59:47.297079 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c160f3-15f2-419b-8c40-3f2a28d45c6c admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:59:47.300378 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 647/1302] 10.222.0.59 () {60 vars in 1144 bytes} [Tue Oct 11 11:59:47 2022] POST /identity/v3/users => generated 299 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 11:59:47.303698 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe7bc74-1300-4a9e-8cc9-202bf6ed96ff admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:59:47.306542 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e52bf0-7de5-4c1b-92ff-d9957bf2f072 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:59:47.307214 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e52bf0-7de5-4c1b-92ff-d9957bf2f072 admin admin] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:47.307496 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e52bf0-7de5-4c1b-92ff-d9957bf2f072 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:47.307778 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e52bf0-7de5-4c1b-92ff-d9957bf2f072 admin admin] PATH_INFO: `/v3/users` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:47.311157 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e52bf0-7de5-4c1b-92ff-d9957bf2f072 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:59:47.313804 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e52bf0-7de5-4c1b-92ff-d9957bf2f072 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:59:47.319899 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe7bc74-1300-4a9e-8cc9-202bf6ed96ff admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:59:47.326328 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b68ff29-1bc2-455a-88c3-c7aa58ad0bbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:59:47.327166 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b68ff29-1bc2-455a-88c3-c7aa58ad0bbc admin admin] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:47.327561 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b68ff29-1bc2-455a-88c3-c7aa58ad0bbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:47.327852 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b68ff29-1bc2-455a-88c3-c7aa58ad0bbc admin admin] PATH_INFO: `/v3/users` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:47.329265 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b68ff29-1bc2-455a-88c3-c7aa58ad0bbc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:59:47.332032 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b68ff29-1bc2-455a-88c3-c7aa58ad0bbc admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:59:47.363905 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 648/1303] 10.222.0.59 () {60 vars in 1144 bytes} [Tue Oct 11 11:59:47 2022] POST /identity/v3/users => generated 299 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 11:59:47.363905 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b68ff29-1bc2-455a-88c3-c7aa58ad0bbc admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:59:47.366016 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 656/1304] 10.222.0.59 () {60 vars in 1144 bytes} [Tue Oct 11 11:59:47 2022] POST /identity/v3/users => generated 309 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 11:59:47.368362 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e52bf0-7de5-4c1b-92ff-d9957bf2f072 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:59:47.382073 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e52bf0-7de5-4c1b-92ff-d9957bf2f072 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:59:47.389100 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd0026e-0544-4777-8335-60870fa6bf1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:59:47.391355 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd0026e-0544-4777-8335-60870fa6bf1c admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:47.391979 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd0026e-0544-4777-8335-60870fa6bf1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:47.392595 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd0026e-0544-4777-8335-60870fa6bf1c admin admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:47.393670 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd0026e-0544-4777-8335-60870fa6bf1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:59:47.396106 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd0026e-0544-4777-8335-60870fa6bf1c admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:59:47.414666 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 657/1305] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 11:59:47 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 11:59:47.417919 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd0026e-0544-4777-8335-60870fa6bf1c admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:59:47.431542 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd0026e-0544-4777-8335-60870fa6bf1c admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:59:47.437198 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22258c7-bf84-4e6e-850e-d1c2c0148a50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:59:47.437998 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22258c7-bf84-4e6e-850e-d1c2c0148a50 admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:47.438403 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22258c7-bf84-4e6e-850e-d1c2c0148a50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:47.438808 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22258c7-bf84-4e6e-850e-d1c2c0148a50 admin admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:47.439604 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22258c7-bf84-4e6e-850e-d1c2c0148a50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:59:47.441843 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22258c7-bf84-4e6e-850e-d1c2c0148a50 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:59:47.443864 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b68ff29-1bc2-455a-88c3-c7aa58ad0bbc admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:59:47.449277 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8fc989-202a-4018-909a-f80fe26dbbca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b1b651c73664d08accc6edf2f79bace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:59:47.450177 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8fc989-202a-4018-909a-f80fe26dbbca service cinder] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:47.450681 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8fc989-202a-4018-909a-f80fe26dbbca service cinder] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:47.451787 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8fc989-202a-4018-909a-f80fe26dbbca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:47.460189 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 658/1306] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 11:59:47 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 11:59:47.461300 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22258c7-bf84-4e6e-850e-d1c2c0148a50 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:59:47.486489 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22258c7-bf84-4e6e-850e-d1c2c0148a50 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:59:47.487531 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8fc989-202a-4018-909a-f80fe26dbbca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:59:47.489521 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8fc989-202a-4018-909a-f80fe26dbbca service cinder] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:59:47.495028 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29cfe5f-298e-4b97-a911-f3e30ad552a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:59:47.496375 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29cfe5f-298e-4b97-a911-f3e30ad552a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:47.496375 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29cfe5f-298e-4b97-a911-f3e30ad552a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:47.497392 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29cfe5f-298e-4b97-a911-f3e30ad552a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:47.497529 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29cfe5f-298e-4b97-a911-f3e30ad552a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:59:47.500169 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29cfe5f-298e-4b97-a911-f3e30ad552a1 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:59:47.515622 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 659/1307] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 11:59:47 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 11:59:47.519240 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 649/1308] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 11:59:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 11:59:47.519240 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8fc989-202a-4018-909a-f80fe26dbbca service cinder] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:59:47.535644 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29cfe5f-298e-4b97-a911-f3e30ad552a1 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:59:47.536721 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8fc989-202a-4018-909a-f80fe26dbbca service cinder] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:59:47.545827 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d402ff2-35f0-4550-ad90-86dfe7dc00a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:59:47.547548 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d402ff2-35f0-4550-ad90-86dfe7dc00a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:47.548182 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d402ff2-35f0-4550-ad90-86dfe7dc00a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:47.549040 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d402ff2-35f0-4550-ad90-86dfe7dc00a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:47.549627 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d402ff2-35f0-4550-ad90-86dfe7dc00a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:59:47.552592 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d402ff2-35f0-4550-ad90-86dfe7dc00a8 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:59:47.561634 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29cfe5f-298e-4b97-a911-f3e30ad552a1 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:59:47.572336 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 650/1309] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 11:59:47 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 11:59:47.572336 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d402ff2-35f0-4550-ad90-86dfe7dc00a8 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:59:47.585204 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d402ff2-35f0-4550-ad90-86dfe7dc00a8 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:59:47.591372 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024cabbc-3eb8-45fd-928a-d448da01d42d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:59:47.592164 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024cabbc-3eb8-45fd-928a-d448da01d42d admin admin] REQUEST_METHOD: `PUT` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:47.592620 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024cabbc-3eb8-45fd-928a-d448da01d42d admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:47.592955 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024cabbc-3eb8-45fd-928a-d448da01d42d admin admin] PATH_INFO: `/v3/projects/5a87190cffa1499f901d96e37abf7190/users/52485eaed13c41b49e16a503b557e138/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:47.598212 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295f97e2-9366-4172-9fc2-ee8928921f50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:59:47.599096 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295f97e2-9366-4172-9fc2-ee8928921f50 admin admin] REQUEST_METHOD: `PUT` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:47.599359 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295f97e2-9366-4172-9fc2-ee8928921f50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:47.599628 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295f97e2-9366-4172-9fc2-ee8928921f50 admin admin] PATH_INFO: `/v3/projects/c3338a5037d74001ac918f972e019fc9/users/4ee624a12fad45da8a00a844ef136626/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:47.616656 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024cabbc-3eb8-45fd-928a-d448da01d42d admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a87190cffa1499f901d96e37abf7190, user_id=52485eaed13c41b49e16a503b557e138, role_id=5cf1f4bd2fef46ab9c6eff56d52f9f69)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:59:47.619830 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024cabbc-3eb8-45fd-928a-d448da01d42d admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:59:47.633085 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295f97e2-9366-4172-9fc2-ee8928921f50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3338a5037d74001ac918f972e019fc9, user_id=4ee624a12fad45da8a00a844ef136626, role_id=5cf1f4bd2fef46ab9c6eff56d52f9f69)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:59:47.638057 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295f97e2-9366-4172-9fc2-ee8928921f50 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:59:47.653768 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 651/1310] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 11:59:47 2022] PUT /identity/v3/projects/5a87190cffa1499f901d96e37abf7190/users/52485eaed13c41b49e16a503b557e138/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 11:59:47.654963 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024cabbc-3eb8-45fd-928a-d448da01d42d admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:59:47.668320 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 660/1311] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 11:59:47 2022] PUT /identity/v3/projects/c3338a5037d74001ac918f972e019fc9/users/4ee624a12fad45da8a00a844ef136626/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 11:59:47.669269 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295f97e2-9366-4172-9fc2-ee8928921f50 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:59:47.786252 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295f97e2-9366-4172-9fc2-ee8928921f50 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:59:47.791944 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec6af77-150e-453d-bf81-82a1943cbeac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:59:47.792717 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec6af77-150e-453d-bf81-82a1943cbeac admin admin] REQUEST_METHOD: `PUT` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:47.793929 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec6af77-150e-453d-bf81-82a1943cbeac admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:47.794231 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec6af77-150e-453d-bf81-82a1943cbeac admin admin] PATH_INFO: `/v3/projects/74f3246047bd45cda1eaa9c0cd8137da/users/69f7ffef575c491e83c7c5f1887fa01f/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:47.805074 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024cabbc-3eb8-45fd-928a-d448da01d42d admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:59:47.816058 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec6af77-150e-453d-bf81-82a1943cbeac admin admin] RBAC: Authorizing `identity:create_grant(project_id=74f3246047bd45cda1eaa9c0cd8137da, user_id=69f7ffef575c491e83c7c5f1887fa01f, role_id=5cf1f4bd2fef46ab9c6eff56d52f9f69)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:59:47.819622 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec6af77-150e-453d-bf81-82a1943cbeac admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:59:47.821854 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b01e072-875f-43b1-90d7-1d2770aa05f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:59:47.822592 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b01e072-875f-43b1-90d7-1d2770aa05f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:47.822908 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b01e072-875f-43b1-90d7-1d2770aa05f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:47.823186 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b01e072-875f-43b1-90d7-1d2770aa05f8 admin admin] PATH_INFO: `/v3/projects/3dac33acae7c4016b680cf0bd68ecfee/users/62b819cfe3e24b37af1d16617a8095af/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:47.841556 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 661/1312] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 11:59:47 2022] PUT /identity/v3/projects/74f3246047bd45cda1eaa9c0cd8137da/users/69f7ffef575c491e83c7c5f1887fa01f/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 11:59:47.846844 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec6af77-150e-453d-bf81-82a1943cbeac admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:59:47.852385 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b01e072-875f-43b1-90d7-1d2770aa05f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3dac33acae7c4016b680cf0bd68ecfee, user_id=62b819cfe3e24b37af1d16617a8095af, role_id=5cf1f4bd2fef46ab9c6eff56d52f9f69)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:59:47.855167 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b01e072-875f-43b1-90d7-1d2770aa05f8 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:59:47.882700 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 652/1313] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 11:59:47 2022] PUT /identity/v3/projects/3dac33acae7c4016b680cf0bd68ecfee/users/62b819cfe3e24b37af1d16617a8095af/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 11:59:47.884351 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b01e072-875f-43b1-90d7-1d2770aa05f8 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:59:47.939242 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:59:48.039616 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b01e072-875f-43b1-90d7-1d2770aa05f8 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:59:48.045615 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:59:48.054115 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f740039-2d10-4ccd-a847-054bfd5515c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:59:48.054808 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f740039-2d10-4ccd-a847-054bfd5515c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:48.055133 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f740039-2d10-4ccd-a847-054bfd5515c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:48.055431 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f740039-2d10-4ccd-a847-054bfd5515c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:48.056100 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f740039-2d10-4ccd-a847-054bfd5515c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:59:48.057736 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f740039-2d10-4ccd-a847-054bfd5515c8 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:59:48.069809 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 653/1314] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 11:59:47 2022] GET /identity/v3/roles => generated 1420 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 11:59:48.074691 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f740039-2d10-4ccd-a847-054bfd5515c8 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:59:48.087147 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f740039-2d10-4ccd-a847-054bfd5515c8 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:59:48.093087 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0830080-5118-4792-affe-51aea1b897e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:59:48.093811 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0830080-5118-4792-affe-51aea1b897e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:48.094150 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0830080-5118-4792-affe-51aea1b897e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:48.094435 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0830080-5118-4792-affe-51aea1b897e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:48.095204 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0830080-5118-4792-affe-51aea1b897e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:59:48.096930 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0830080-5118-4792-affe-51aea1b897e4 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:59:48.113137 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 654/1315] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 11:59:48 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 11:59:48.113137 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0830080-5118-4792-affe-51aea1b897e4 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:59:48.125830 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0830080-5118-4792-affe-51aea1b897e4 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:59:48.138164 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cebdf22-1956-416f-8586-a9d97d988e37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:59:48.139069 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cebdf22-1956-416f-8586-a9d97d988e37 admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:48.142239 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cebdf22-1956-416f-8586-a9d97d988e37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:48.144112 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cebdf22-1956-416f-8586-a9d97d988e37 admin admin] PATH_INFO: `/v3/roles` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:48.145863 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cebdf22-1956-416f-8586-a9d97d988e37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:59:48.148489 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec6af77-150e-453d-bf81-82a1943cbeac admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:59:48.149246 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cebdf22-1956-416f-8586-a9d97d988e37 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:59:48.166752 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 655/1316] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 11:59:48 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 11:59:48.168423 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cebdf22-1956-416f-8586-a9d97d988e37 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:59:48.172221 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb5c53c-9e14-4f36-b2fc-3f449f5df030 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:59:48.173271 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb5c53c-9e14-4f36-b2fc-3f449f5df030 admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:48.173751 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb5c53c-9e14-4f36-b2fc-3f449f5df030 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:48.174281 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb5c53c-9e14-4f36-b2fc-3f449f5df030 admin admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:48.175850 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb5c53c-9e14-4f36-b2fc-3f449f5df030 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:59:48.180705 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb5c53c-9e14-4f36-b2fc-3f449f5df030 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:59:48.183073 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cebdf22-1956-416f-8586-a9d97d988e37 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:59:48.189993 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b63fc8b-e643-4ea4-8f87-9943a46516a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:59:48.190918 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b63fc8b-e643-4ea4-8f87-9943a46516a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:48.191299 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b63fc8b-e643-4ea4-8f87-9943a46516a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:48.191723 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b63fc8b-e643-4ea4-8f87-9943a46516a1 admin admin] PATH_INFO: `/v3/projects/c3338a5037d74001ac918f972e019fc9/users/4ee624a12fad45da8a00a844ef136626/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:48.194359 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b63fc8b-e643-4ea4-8f87-9943a46516a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3338a5037d74001ac918f972e019fc9, user_id=4ee624a12fad45da8a00a844ef136626, role_id=5cf1f4bd2fef46ab9c6eff56d52f9f69)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:59:48.197865 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b63fc8b-e643-4ea4-8f87-9943a46516a1 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:59:48.221105 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 662/1318] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 11:59:47 2022] GET /identity/v3/roles => generated 1420 bytes in 376 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 11:59:48.221105 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb5c53c-9e14-4f36-b2fc-3f449f5df030 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:59:48.221901 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 656/1318] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 11:59:48 2022] PUT /identity/v3/projects/c3338a5037d74001ac918f972e019fc9/users/4ee624a12fad45da8a00a844ef136626/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 11:59:48.223137 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b63fc8b-e643-4ea4-8f87-9943a46516a1 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:59:48.242128 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb5c53c-9e14-4f36-b2fc-3f449f5df030 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:59:48.307365 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b63fc8b-e643-4ea4-8f87-9943a46516a1 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:59:48.313794 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfa03d8-4a82-44e9-829a-47a9fad57c7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:59:48.314756 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfa03d8-4a82-44e9-829a-47a9fad57c7c admin admin] REQUEST_METHOD: `PUT` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:48.315100 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfa03d8-4a82-44e9-829a-47a9fad57c7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:48.315817 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfa03d8-4a82-44e9-829a-47a9fad57c7c admin admin] PATH_INFO: `/v3/projects/3dac33acae7c4016b680cf0bd68ecfee/users/62b819cfe3e24b37af1d16617a8095af/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:48.317643 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfa03d8-4a82-44e9-829a-47a9fad57c7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=3dac33acae7c4016b680cf0bd68ecfee, user_id=62b819cfe3e24b37af1d16617a8095af, role_id=5cf1f4bd2fef46ab9c6eff56d52f9f69)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:59:48.321287 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfa03d8-4a82-44e9-829a-47a9fad57c7c admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:59:48.334903 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 657/1319] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 11:59:48 2022] PUT /identity/v3/projects/3dac33acae7c4016b680cf0bd68ecfee/users/62b819cfe3e24b37af1d16617a8095af/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 11:59:48.335812 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfa03d8-4a82-44e9-829a-47a9fad57c7c admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:59:48.380838 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd5cdd4-e64b-4aef-9eea-d130c534244c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:59:48.381943 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd5cdd4-e64b-4aef-9eea-d130c534244c admin admin] REQUEST_METHOD: `PUT` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:48.382425 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd5cdd4-e64b-4aef-9eea-d130c534244c admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:48.382930 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd5cdd4-e64b-4aef-9eea-d130c534244c admin admin] PATH_INFO: `/v3/projects/74f3246047bd45cda1eaa9c0cd8137da/users/69f7ffef575c491e83c7c5f1887fa01f/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:48.388979 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd5cdd4-e64b-4aef-9eea-d130c534244c admin admin] RBAC: Authorizing `identity:create_grant(project_id=74f3246047bd45cda1eaa9c0cd8137da, user_id=69f7ffef575c491e83c7c5f1887fa01f, role_id=5cf1f4bd2fef46ab9c6eff56d52f9f69)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:59:48.397436 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd5cdd4-e64b-4aef-9eea-d130c534244c admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:59:48.422115 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 663/1320] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 11:59:48 2022] PUT /identity/v3/projects/74f3246047bd45cda1eaa9c0cd8137da/users/69f7ffef575c491e83c7c5f1887fa01f/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 11:59:48.423688 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d236720f-48c1-4483-bb4f-b195a23c6d24 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:48.424024 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d236720f-48c1-4483-bb4f-b195a23c6d24 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:48.424412 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d236720f-48c1-4483-bb4f-b195a23c6d24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:48.470925 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-d236720f-48c1-4483-bb4f-b195a23c6d24 None None] MFA Rules not processed for user `4ee624a12fad45da8a00a844ef136626`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:59:48.472231 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-d236720f-48c1-4483-bb4f-b195a23c6d24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:59:48.509929 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfa03d8-4a82-44e9-829a-47a9fad57c7c admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:59:48.545828 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:59:48.589869 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 664/1321] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 11:59:48 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 11:59:48.591516 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8a508f-63f7-4d8d-862b-72c7c23b9861 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:48.591805 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8a508f-63f7-4d8d-862b-72c7c23b9861 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:48.592082 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8a508f-63f7-4d8d-862b-72c7c23b9861 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:48.632397 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-fb8a508f-63f7-4d8d-862b-72c7c23b9861 None None] MFA Rules not processed for user `62b819cfe3e24b37af1d16617a8095af`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:59:48.634536 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-fb8a508f-63f7-4d8d-862b-72c7c23b9861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:59:48.648522 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e111d51d-2857-4b3c-9fd4-415a7244a729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:59:48.649671 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e111d51d-2857-4b3c-9fd4-415a7244a729 admin admin] REQUEST_METHOD: `PUT` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:48.650419 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e111d51d-2857-4b3c-9fd4-415a7244a729 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:48.650895 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e111d51d-2857-4b3c-9fd4-415a7244a729 admin admin] PATH_INFO: `/v3/projects/5a87190cffa1499f901d96e37abf7190/users/52485eaed13c41b49e16a503b557e138/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:48.654934 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e111d51d-2857-4b3c-9fd4-415a7244a729 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a87190cffa1499f901d96e37abf7190, user_id=52485eaed13c41b49e16a503b557e138, role_id=5cf1f4bd2fef46ab9c6eff56d52f9f69)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:59:48.658838 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e111d51d-2857-4b3c-9fd4-415a7244a729 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:59:48.675725 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 658/1322] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 11:59:48 2022] PUT /identity/v3/projects/5a87190cffa1499f901d96e37abf7190/users/52485eaed13c41b49e16a503b557e138/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 11:59:48.677297 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4157b6-6bd2-401f-8bb9-4f089b3dd008 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:48.677649 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4157b6-6bd2-401f-8bb9-4f089b3dd008 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:48.678020 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4157b6-6bd2-401f-8bb9-4f089b3dd008 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:48.707466 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:59:48.715656 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-9e4157b6-6bd2-401f-8bb9-4f089b3dd008 None None] MFA Rules not processed for user `69f7ffef575c491e83c7c5f1887fa01f`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:59:48.717179 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-9e4157b6-6bd2-401f-8bb9-4f089b3dd008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:59:48.745227 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 665/1323] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 11:59:48 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 11:59:48.749991 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8a508f-63f7-4d8d-862b-72c7c23b9861 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:59:48.829430 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 659/1324] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 11:59:48 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 11:59:48.829430 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d1f1d2-e7ce-4bf1-a4a8-d7ff3b1024a8 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:48.831907 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d1f1d2-e7ce-4bf1-a4a8-d7ff3b1024a8 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:48.832316 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d1f1d2-e7ce-4bf1-a4a8-d7ff3b1024a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:48.867610 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8a508f-63f7-4d8d-862b-72c7c23b9861 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:59:48.917637 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-99d1f1d2-e7ce-4bf1-a4a8-d7ff3b1024a8 None None] MFA Rules not processed for user `52485eaed13c41b49e16a503b557e138`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:59:48.919173 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-99d1f1d2-e7ce-4bf1-a4a8-d7ff3b1024a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:59:49.001805 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3b2523-fa94-4b37-a53f-b7e054882629 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b1b651c73664d08accc6edf2f79bace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:59:49.003506 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3b2523-fa94-4b37-a53f-b7e054882629 service cinder] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:49.004300 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3b2523-fa94-4b37-a53f-b7e054882629 service cinder] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:49.004926 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3b2523-fa94-4b37-a53f-b7e054882629 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:49.123898 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 660/1325] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 11:59:48 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 11:59:49.129324 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d1f1d2-e7ce-4bf1-a4a8-d7ff3b1024a8 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:59:49.151805 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d1f1d2-e7ce-4bf1-a4a8-d7ff3b1024a8 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:59:49.162997 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab34297f-8f59-4c85-8970-f5ed7a1ee757 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b1b651c73664d08accc6edf2f79bace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:59:49.164992 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab34297f-8f59-4c85-8970-f5ed7a1ee757 service cinder] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:49.166196 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab34297f-8f59-4c85-8970-f5ed7a1ee757 service cinder] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:49.168704 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab34297f-8f59-4c85-8970-f5ed7a1ee757 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:49.181056 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3b2523-fa94-4b37-a53f-b7e054882629 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:59:49.183740 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3b2523-fa94-4b37-a53f-b7e054882629 service cinder] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:59:49.211713 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 666/1326] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 11:59:48 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 11:59:49.223709 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:59:49.307915 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab34297f-8f59-4c85-8970-f5ed7a1ee757 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:59:49.311694 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab34297f-8f59-4c85-8970-f5ed7a1ee757 service cinder] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:59:49.337366 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 661/1327] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 11:59:49 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 11:59:49.449182 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3b2523-fa94-4b37-a53f-b7e054882629 service cinder] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:59:49.471090 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3b2523-fa94-4b37-a53f-b7e054882629 service cinder] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:59:49.480045 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6f1f4d-5f5f-4fbf-b2d8-70e70c2b1df8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b1b651c73664d08accc6edf2f79bace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:59:49.480695 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6f1f4d-5f5f-4fbf-b2d8-70e70c2b1df8 service cinder] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:49.480979 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6f1f4d-5f5f-4fbf-b2d8-70e70c2b1df8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:49.481257 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6f1f4d-5f5f-4fbf-b2d8-70e70c2b1df8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:49.507817 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6f1f4d-5f5f-4fbf-b2d8-70e70c2b1df8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:59:49.509742 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6f1f4d-5f5f-4fbf-b2d8-70e70c2b1df8 service cinder] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:59:49.535921 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 667/1328] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 11:59:49 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 11 11:59:49.618446 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c688c17-be09-40c3-af87-e1172f1564c2 None None] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:49.619993 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c688c17-be09-40c3-af87-e1172f1564c2 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:49.620845 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c688c17-be09-40c3-af87-e1172f1564c2 None None] PATH_INFO: `/` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:49.627164 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 662/1329] 10.222.0.59 () {54 vars in 961 bytes} [Tue Oct 11 11:59:49 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:59:49.630081 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6f1f4d-5f5f-4fbf-b2d8-70e70c2b1df8 service cinder] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:59:49.643257 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6f1f4d-5f5f-4fbf-b2d8-70e70c2b1df8 service cinder] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:59:49.649630 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e49ad2-584b-4ce6-9c7d-f5a6e81bb2f9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b1b651c73664d08accc6edf2f79bace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:59:49.650278 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e49ad2-584b-4ce6-9c7d-f5a6e81bb2f9 service cinder] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:49.650587 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e49ad2-584b-4ce6-9c7d-f5a6e81bb2f9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:49.650856 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e49ad2-584b-4ce6-9c7d-f5a6e81bb2f9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:49.654789 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681fa907-85da-444e-ab77-6f9d96524c33 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:49.655535 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681fa907-85da-444e-ab77-6f9d96524c33 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:49.656203 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681fa907-85da-444e-ab77-6f9d96524c33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:49.678166 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e49ad2-584b-4ce6-9c7d-f5a6e81bb2f9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:59:49.679570 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e49ad2-584b-4ce6-9c7d-f5a6e81bb2f9 service cinder] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:59:49.710922 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 668/1330] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 11:59:49 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 11:59:49.779442 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-681fa907-85da-444e-ab77-6f9d96524c33 None None] MFA Rules not processed for user `bad5e1cb306b46818096940acc3810b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:59:49.785688 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-681fa907-85da-444e-ab77-6f9d96524c33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:59:49.811446 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e49ad2-584b-4ce6-9c7d-f5a6e81bb2f9 service cinder] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:59:49.876780 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:59:49.961079 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e49ad2-584b-4ce6-9c7d-f5a6e81bb2f9 service cinder] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:59:49.970620 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b396b3-2a42-4d1e-8db1-eeeb16b42d5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:59:49.971324 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b396b3-2a42-4d1e-8db1-eeeb16b42d5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:49.971605 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b396b3-2a42-4d1e-8db1-eeeb16b42d5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:49.971885 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b396b3-2a42-4d1e-8db1-eeeb16b42d5c admin admin] PATH_INFO: `/v3/users/37e06773cb10450cb28d42d98f7c9f21` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:49.975740 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b396b3-2a42-4d1e-8db1-eeeb16b42d5c admin admin] RBAC: Authorizing `identity:delete_user(user_id=37e06773cb10450cb28d42d98f7c9f21)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:59:49.977946 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b396b3-2a42-4d1e-8db1-eeeb16b42d5c admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:59:50.106942 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 663/1331] 10.222.0.59 () {58 vars in 1058 bytes} [Tue Oct 11 11:59:49 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 456 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 11:59:50.108215 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681fa907-85da-444e-ab77-6f9d96524c33 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:59:50.136251 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681fa907-85da-444e-ab77-6f9d96524c33 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:59:50.156487 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d057147f-7907-4280-8148-ae111f7f1b48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:59:50.161574 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d057147f-7907-4280-8148-ae111f7f1b48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:50.161942 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d057147f-7907-4280-8148-ae111f7f1b48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:50.162287 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d057147f-7907-4280-8148-ae111f7f1b48 admin admin] PATH_INFO: `/v3/users/9c3e056a705e4d19972f77f507c7f95c` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:50.165017 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d057147f-7907-4280-8148-ae111f7f1b48 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c3e056a705e4d19972f77f507c7f95c)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:59:50.166237 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: WARNING py.warnings [None req-d057147f-7907-4280-8148-ae111f7f1b48 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 11 11:59:50.166237 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: warnings.warn(msg) Oct 11 11:59:50.166237 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: Oct 11 11:59:50.169272 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d057147f-7907-4280-8148-ae111f7f1b48 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:59:50.183745 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-17b396b3-2a42-4d1e-8db1-eeeb16b42d5c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '37e06773cb10450cb28d42d98f7c9f21'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 11:59:50.223691 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-17b396b3-2a42-4d1e-8db1-eeeb16b42d5c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '37e06773cb10450cb28d42d98f7c9f21'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 11:59:50.226166 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-17b396b3-2a42-4d1e-8db1-eeeb16b42d5c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '37e06773cb10450cb28d42d98f7c9f21'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 11:59:50.246755 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-17b396b3-2a42-4d1e-8db1-eeeb16b42d5c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '37e06773cb10450cb28d42d98f7c9f21'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 11:59:50.248068 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-17b396b3-2a42-4d1e-8db1-eeeb16b42d5c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '37e06773cb10450cb28d42d98f7c9f21'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 11:59:50.268362 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 669/1332] 10.222.0.59 () {58 vars in 1224 bytes} [Tue Oct 11 11:59:49 2022] DELETE /identity/v3/users/37e06773cb10450cb28d42d98f7c9f21 => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 11:59:50.270365 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09958dbb-4fe3-450f-914f-20720bb13cbf None None] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:50.270743 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09958dbb-4fe3-450f-914f-20720bb13cbf None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:50.271130 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09958dbb-4fe3-450f-914f-20720bb13cbf None None] PATH_INFO: `/` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:50.283390 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 670/1333] 10.222.0.59 () {54 vars in 912 bytes} [Tue Oct 11 11:59:50 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 11:59:50.284530 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09958dbb-4fe3-450f-914f-20720bb13cbf None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:59:50.287536 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-09958dbb-4fe3-450f-914f-20720bb13cbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:59:50.314250 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-d057147f-7907-4280-8148-ae111f7f1b48 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c3e056a705e4d19972f77f507c7f95c'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 11:59:50.343253 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-d057147f-7907-4280-8148-ae111f7f1b48 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c3e056a705e4d19972f77f507c7f95c'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 11:59:50.358664 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-d057147f-7907-4280-8148-ae111f7f1b48 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c3e056a705e4d19972f77f507c7f95c'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 11:59:50.360786 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-d057147f-7907-4280-8148-ae111f7f1b48 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c3e056a705e4d19972f77f507c7f95c'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 11:59:50.375457 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-d057147f-7907-4280-8148-ae111f7f1b48 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c3e056a705e4d19972f77f507c7f95c'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 11:59:50.387772 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 664/1334] 10.222.0.59 () {58 vars in 1224 bytes} [Tue Oct 11 11:59:50 2022] DELETE /identity/v3/users/9c3e056a705e4d19972f77f507c7f95c => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 11:59:50.389425 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280ec83b-b32d-4a66-9a38-cd2ff5c6a293 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:50.389724 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280ec83b-b32d-4a66-9a38-cd2ff5c6a293 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:50.390007 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280ec83b-b32d-4a66-9a38-cd2ff5c6a293 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:50.399918 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09958dbb-4fe3-450f-914f-20720bb13cbf None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:59:50.413829 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-280ec83b-b32d-4a66-9a38-cd2ff5c6a293 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:59:50.415590 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-280ec83b-b32d-4a66-9a38-cd2ff5c6a293 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:59:50.447783 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:59:50.497675 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:59:50.524082 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 665/1335] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 11:59:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 11:59:50.526451 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d713ce-8208-47f6-b2e8-ca98071aa2b4 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:50.527079 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d713ce-8208-47f6-b2e8-ca98071aa2b4 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:50.527498 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d713ce-8208-47f6-b2e8-ca98071aa2b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:50.543700 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f6237b-f0ff-4445-909d-b85dee4d68bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:59:50.544839 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f6237b-f0ff-4445-909d-b85dee4d68bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:50.545338 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f6237b-f0ff-4445-909d-b85dee4d68bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:50.545839 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f6237b-f0ff-4445-909d-b85dee4d68bd admin admin] PATH_INFO: `/v3/users/449cae8c746b47a9bbedec9a845cb9bc` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:50.551020 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f6237b-f0ff-4445-909d-b85dee4d68bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=449cae8c746b47a9bbedec9a845cb9bc)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:59:50.555319 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f6237b-f0ff-4445-909d-b85dee4d68bd admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:59:50.555691 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-42d713ce-8208-47f6-b2e8-ca98071aa2b4 None None] MFA Rules not processed for user `bad5e1cb306b46818096940acc3810b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:59:50.557177 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-42d713ce-8208-47f6-b2e8-ca98071aa2b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:59:50.675731 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-f6f6237b-f0ff-4445-909d-b85dee4d68bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '449cae8c746b47a9bbedec9a845cb9bc'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 11:59:50.683244 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 666/1336] 10.222.0.59 () {58 vars in 1009 bytes} [Tue Oct 11 11:59:50 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 11:59:50.687421 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878653c4-bacd-4154-9cab-e06458f6177e None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:50.688727 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878653c4-bacd-4154-9cab-e06458f6177e None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:50.689399 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878653c4-bacd-4154-9cab-e06458f6177e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:50.722067 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-f6f6237b-f0ff-4445-909d-b85dee4d68bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '449cae8c746b47a9bbedec9a845cb9bc'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 11:59:50.728757 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-f6f6237b-f0ff-4445-909d-b85dee4d68bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '449cae8c746b47a9bbedec9a845cb9bc'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 11:59:50.731300 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-878653c4-bacd-4154-9cab-e06458f6177e None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:59:50.733118 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-878653c4-bacd-4154-9cab-e06458f6177e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:59:50.747391 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 667/1337] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 11:59:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 11:59:50.748939 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878653c4-bacd-4154-9cab-e06458f6177e None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:59:50.750901 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-878653c4-bacd-4154-9cab-e06458f6177e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:59:50.757395 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-f6f6237b-f0ff-4445-909d-b85dee4d68bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '449cae8c746b47a9bbedec9a845cb9bc'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 11:59:50.758046 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-f6f6237b-f0ff-4445-909d-b85dee4d68bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '449cae8c746b47a9bbedec9a845cb9bc'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 11:59:50.781688 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 671/1338] 10.222.0.59 () {58 vars in 1224 bytes} [Tue Oct 11 11:59:50 2022] DELETE /identity/v3/users/449cae8c746b47a9bbedec9a845cb9bc => generated 0 bytes in 498 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 11:59:50.782912 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f6237b-f0ff-4445-909d-b85dee4d68bd admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:59:50.792484 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-f6f6237b-f0ff-4445-909d-b85dee4d68bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:59:50.839298 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878653c4-bacd-4154-9cab-e06458f6177e None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:59:50.921097 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b271af5-8199-4efc-8e14-961cc02f6bcf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e563fe174db4cc19ae6bf8162cc3d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:59:50.922205 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b271af5-8199-4efc-8e14-961cc02f6bcf service neutron] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:50.922720 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f6237b-f0ff-4445-909d-b85dee4d68bd admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:59:50.923141 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b271af5-8199-4efc-8e14-961cc02f6bcf service neutron] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:50.923623 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b271af5-8199-4efc-8e14-961cc02f6bcf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:50.927393 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-1b271af5-8199-4efc-8e14-961cc02f6bcf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:59:50.930595 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d7c3ed-7ea2-4be2-b9b1-696271aca62f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b1b651c73664d08accc6edf2f79bace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:59:50.931244 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d7c3ed-7ea2-4be2-b9b1-696271aca62f service cinder] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:50.931529 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d7c3ed-7ea2-4be2-b9b1-696271aca62f service cinder] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:50.931808 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d7c3ed-7ea2-4be2-b9b1-696271aca62f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:50.940555 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-37d7c3ed-7ea2-4be2-b9b1-696271aca62f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:59:50.964500 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:59:51.036196 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b271af5-8199-4efc-8e14-961cc02f6bcf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:59:51.039860 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b271af5-8199-4efc-8e14-961cc02f6bcf service neutron] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:59:51.055413 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 668/1339] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 11:59:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 11:59:51.057919 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b801a0e8-6712-4145-af97-fc0b50e3ef1c None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:51.058324 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b801a0e8-6712-4145-af97-fc0b50e3ef1c None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:51.058934 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b801a0e8-6712-4145-af97-fc0b50e3ef1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:51.085862 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-b801a0e8-6712-4145-af97-fc0b50e3ef1c None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:59:51.088579 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-b801a0e8-6712-4145-af97-fc0b50e3ef1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:59:51.099483 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d7c3ed-7ea2-4be2-b9b1-696271aca62f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:59:51.101095 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d7c3ed-7ea2-4be2-b9b1-696271aca62f service cinder] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:59:51.104576 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 669/1340] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 11:59:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 11:59:51.106930 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b801a0e8-6712-4145-af97-fc0b50e3ef1c None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:59:51.126137 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 672/1341] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 11:59:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 11:59:51.130106 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b801a0e8-6712-4145-af97-fc0b50e3ef1c None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:59:51.134860 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5681018-d02a-4afa-9d90-3bb7d56b4da8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e563fe174db4cc19ae6bf8162cc3d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:59:51.135691 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5681018-d02a-4afa-9d90-3bb7d56b4da8 service neutron] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:51.136142 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5681018-d02a-4afa-9d90-3bb7d56b4da8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:51.136576 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5681018-d02a-4afa-9d90-3bb7d56b4da8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:51.159766 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5681018-d02a-4afa-9d90-3bb7d56b4da8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:59:51.161533 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5681018-d02a-4afa-9d90-3bb7d56b4da8 service neutron] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:59:51.189404 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 670/1342] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 11:59:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 11:59:51.193240 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d7c3ed-7ea2-4be2-b9b1-696271aca62f service cinder] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:59:51.217049 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d7c3ed-7ea2-4be2-b9b1-696271aca62f service cinder] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:59:51.221827 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e42548-4a0c-4a7e-9839-bc6634c75011 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e563fe174db4cc19ae6bf8162cc3d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:59:51.222177 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e42548-4a0c-4a7e-9839-bc6634c75011 service neutron] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:51.222549 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e42548-4a0c-4a7e-9839-bc6634c75011 service neutron] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:51.223100 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e42548-4a0c-4a7e-9839-bc6634c75011 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:51.236682 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e42548-4a0c-4a7e-9839-bc6634c75011 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:59:51.239385 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e42548-4a0c-4a7e-9839-bc6634c75011 service neutron] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:59:51.244935 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:59:51.255781 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 673/1343] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 11:59:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 11:59:53.021404 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5681018-d02a-4afa-9d90-3bb7d56b4da8 service neutron] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:59:53.023460 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-a5681018-d02a-4afa-9d90-3bb7d56b4da8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:59:53.058727 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5681018-d02a-4afa-9d90-3bb7d56b4da8 service neutron] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:59:53.061521 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e42548-4a0c-4a7e-9839-bc6634c75011 service neutron] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:59:53.068166 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-82e42548-4a0c-4a7e-9839-bc6634c75011 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:59:53.071077 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f6896f-a45a-401b-80fe-b65cb0f1dc18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:59:53.072828 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f6896f-a45a-401b-80fe-b65cb0f1dc18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:53.073091 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f6896f-a45a-401b-80fe-b65cb0f1dc18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:53.074099 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f6896f-a45a-401b-80fe-b65cb0f1dc18 admin admin] PATH_INFO: `/v3/projects/a5e9e0788b5c42c9837666a2403c87cc` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:53.076954 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f6896f-a45a-401b-80fe-b65cb0f1dc18 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5e9e0788b5c42c9837666a2403c87cc)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:59:53.089166 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f6896f-a45a-401b-80fe-b65cb0f1dc18 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:59:53.107857 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e42548-4a0c-4a7e-9839-bc6634c75011 service neutron] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:59:53.124089 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4bfe2b-2346-41ad-93f4-a2ffcbf99bf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:59:53.126091 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4bfe2b-2346-41ad-93f4-a2ffcbf99bf3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:53.127844 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4bfe2b-2346-41ad-93f4-a2ffcbf99bf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:53.128552 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4bfe2b-2346-41ad-93f4-a2ffcbf99bf3 admin admin] PATH_INFO: `/v3/projects/c2e9fa99631f4b0b9c3cfe2cf33a8a82` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:53.131313 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4bfe2b-2346-41ad-93f4-a2ffcbf99bf3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2e9fa99631f4b0b9c3cfe2cf33a8a82)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:59:53.134517 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: WARNING py.warnings [None req-8f4bfe2b-2346-41ad-93f4-a2ffcbf99bf3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 11 11:59:53.134517 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: warnings.warn(msg) Oct 11 11:59:53.134517 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: Oct 11 11:59:53.141936 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4bfe2b-2346-41ad-93f4-a2ffcbf99bf3 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:59:53.232133 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-79f6896f-a45a-401b-80fe-b65cb0f1dc18 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5e9e0788b5c42c9837666a2403c87cc', 'request_id': 'req-79f6896f-a45a-401b-80fe-b65cb0f1dc18'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 11:59:53.245307 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-79f6896f-a45a-401b-80fe-b65cb0f1dc18 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5e9e0788b5c42c9837666a2403c87cc', 'request_id': 'req-79f6896f-a45a-401b-80fe-b65cb0f1dc18'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 11:59:53.268902 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-79f6896f-a45a-401b-80fe-b65cb0f1dc18 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5e9e0788b5c42c9837666a2403c87cc', 'request_id': 'req-79f6896f-a45a-401b-80fe-b65cb0f1dc18'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 11:59:53.289373 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-79f6896f-a45a-401b-80fe-b65cb0f1dc18 admin admin] The token cache is being invalidate because project a5e9e0788b5c42c9837666a2403c87cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 11:59:53.289373 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-79f6896f-a45a-401b-80fe-b65cb0f1dc18 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 11:59:53.290378 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-79f6896f-a45a-401b-80fe-b65cb0f1dc18 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 11:59:53.297665 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 671/1344] 10.222.0.59 () {58 vars in 1233 bytes} [Tue Oct 11 11:59:53 2022] DELETE /identity/v3/projects/a5e9e0788b5c42c9837666a2403c87cc => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 11:59:53.297665 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f6896f-a45a-401b-80fe-b65cb0f1dc18 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:59:53.297665 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-79f6896f-a45a-401b-80fe-b65cb0f1dc18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:59:53.382433 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-8f4bfe2b-2346-41ad-93f4-a2ffcbf99bf3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2e9fa99631f4b0b9c3cfe2cf33a8a82', 'request_id': 'req-8f4bfe2b-2346-41ad-93f4-a2ffcbf99bf3'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 11:59:53.400689 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f6896f-a45a-401b-80fe-b65cb0f1dc18 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:59:53.413012 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-8f4bfe2b-2346-41ad-93f4-a2ffcbf99bf3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2e9fa99631f4b0b9c3cfe2cf33a8a82', 'request_id': 'req-8f4bfe2b-2346-41ad-93f4-a2ffcbf99bf3'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 11:59:53.451257 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-8f4bfe2b-2346-41ad-93f4-a2ffcbf99bf3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2e9fa99631f4b0b9c3cfe2cf33a8a82', 'request_id': 'req-8f4bfe2b-2346-41ad-93f4-a2ffcbf99bf3'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 11:59:53.472361 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:59:53.487500 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-8f4bfe2b-2346-41ad-93f4-a2ffcbf99bf3 admin admin] The token cache is being invalidate because project c2e9fa99631f4b0b9c3cfe2cf33a8a82 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81432) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 11:59:53.487998 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-8f4bfe2b-2346-41ad-93f4-a2ffcbf99bf3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 11:59:53.490016 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-8f4bfe2b-2346-41ad-93f4-a2ffcbf99bf3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 11:59:53.493323 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 674/1345] 10.222.0.59 () {58 vars in 1233 bytes} [Tue Oct 11 11:59:53 2022] DELETE /identity/v3/projects/c2e9fa99631f4b0b9c3cfe2cf33a8a82 => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 11:59:53.497066 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4bfe2b-2346-41ad-93f4-a2ffcbf99bf3 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:59:53.503439 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-8f4bfe2b-2346-41ad-93f4-a2ffcbf99bf3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:59:53.549399 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855d9143-9594-45af-b306-da72faf1664d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:59:53.550074 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855d9143-9594-45af-b306-da72faf1664d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:53.550358 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855d9143-9594-45af-b306-da72faf1664d admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:53.550694 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855d9143-9594-45af-b306-da72faf1664d admin admin] PATH_INFO: `/v3/projects/0b77d55b7953427493baf1ed9f855c83` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:53.555494 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855d9143-9594-45af-b306-da72faf1664d admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b77d55b7953427493baf1ed9f855c83)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:59:53.562769 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855d9143-9594-45af-b306-da72faf1664d admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:59:53.763258 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4bfe2b-2346-41ad-93f4-a2ffcbf99bf3 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:59:53.808999 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-855d9143-9594-45af-b306-da72faf1664d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b77d55b7953427493baf1ed9f855c83', 'request_id': 'req-855d9143-9594-45af-b306-da72faf1664d'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 11:59:53.834685 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-855d9143-9594-45af-b306-da72faf1664d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b77d55b7953427493baf1ed9f855c83', 'request_id': 'req-855d9143-9594-45af-b306-da72faf1664d'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 11:59:53.853636 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-855d9143-9594-45af-b306-da72faf1664d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b77d55b7953427493baf1ed9f855c83', 'request_id': 'req-855d9143-9594-45af-b306-da72faf1664d'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 11:59:53.881104 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-855d9143-9594-45af-b306-da72faf1664d admin admin] The token cache is being invalidate because project 0b77d55b7953427493baf1ed9f855c83 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 11:59:53.883890 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-855d9143-9594-45af-b306-da72faf1664d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 11:59:53.884863 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-855d9143-9594-45af-b306-da72faf1664d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 11:59:53.893630 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 672/1346] 10.222.0.59 () {58 vars in 1233 bytes} [Tue Oct 11 11:59:53 2022] DELETE /identity/v3/projects/0b77d55b7953427493baf1ed9f855c83 => generated 0 bytes in 596 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 11:59:53.894836 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855d9143-9594-45af-b306-da72faf1664d admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:59:53.899883 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-855d9143-9594-45af-b306-da72faf1664d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:59:53.921065 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3f8d36-05cd-4412-babe-5e47df44c225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:59:53.922284 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3f8d36-05cd-4412-babe-5e47df44c225 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:53.923116 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3f8d36-05cd-4412-babe-5e47df44c225 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:53.924023 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3f8d36-05cd-4412-babe-5e47df44c225 admin admin] PATH_INFO: `/v3/users/62b819cfe3e24b37af1d16617a8095af` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:53.926290 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3f8d36-05cd-4412-babe-5e47df44c225 admin admin] RBAC: Authorizing `identity:delete_user(user_id=62b819cfe3e24b37af1d16617a8095af)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:59:53.930387 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3f8d36-05cd-4412-babe-5e47df44c225 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:59:54.072740 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:59:54.102264 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-5b3f8d36-05cd-4412-babe-5e47df44c225 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '62b819cfe3e24b37af1d16617a8095af'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 11:59:54.133545 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-5b3f8d36-05cd-4412-babe-5e47df44c225 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '62b819cfe3e24b37af1d16617a8095af'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 11:59:54.135325 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-5b3f8d36-05cd-4412-babe-5e47df44c225 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '62b819cfe3e24b37af1d16617a8095af'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 11:59:54.135872 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855d9143-9594-45af-b306-da72faf1664d admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:59:54.149192 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f45b7bf-5b7e-4fa9-82cd-ff8c7c86f178 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:59:54.153007 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f45b7bf-5b7e-4fa9-82cd-ff8c7c86f178 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:54.153837 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f45b7bf-5b7e-4fa9-82cd-ff8c7c86f178 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:54.154225 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f45b7bf-5b7e-4fa9-82cd-ff8c7c86f178 admin admin] PATH_INFO: `/v3/users/69f7ffef575c491e83c7c5f1887fa01f` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:54.156392 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f45b7bf-5b7e-4fa9-82cd-ff8c7c86f178 admin admin] RBAC: Authorizing `identity:delete_user(user_id=69f7ffef575c491e83c7c5f1887fa01f)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:59:54.161371 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-5b3f8d36-05cd-4412-babe-5e47df44c225 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '62b819cfe3e24b37af1d16617a8095af'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 11:59:54.163116 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-5b3f8d36-05cd-4412-babe-5e47df44c225 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '62b819cfe3e24b37af1d16617a8095af'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 11:59:54.171612 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f45b7bf-5b7e-4fa9-82cd-ff8c7c86f178 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:59:54.203851 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 675/1347] 10.222.0.59 () {58 vars in 1224 bytes} [Tue Oct 11 11:59:53 2022] DELETE /identity/v3/users/62b819cfe3e24b37af1d16617a8095af => generated 0 bytes in 710 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 11:59:54.204967 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3f8d36-05cd-4412-babe-5e47df44c225 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:59:54.208552 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-5b3f8d36-05cd-4412-babe-5e47df44c225 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:59:54.308689 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-7f45b7bf-5b7e-4fa9-82cd-ff8c7c86f178 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69f7ffef575c491e83c7c5f1887fa01f'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 11:59:54.376744 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:59:54.389341 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-7f45b7bf-5b7e-4fa9-82cd-ff8c7c86f178 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69f7ffef575c491e83c7c5f1887fa01f'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 11:59:54.413083 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-7f45b7bf-5b7e-4fa9-82cd-ff8c7c86f178 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69f7ffef575c491e83c7c5f1887fa01f'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 11:59:54.413718 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-7f45b7bf-5b7e-4fa9-82cd-ff8c7c86f178 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69f7ffef575c491e83c7c5f1887fa01f'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 11:59:54.442024 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-7f45b7bf-5b7e-4fa9-82cd-ff8c7c86f178 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69f7ffef575c491e83c7c5f1887fa01f'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 11:59:54.462542 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 673/1348] 10.222.0.59 () {58 vars in 1224 bytes} [Tue Oct 11 11:59:53 2022] DELETE /identity/v3/users/69f7ffef575c491e83c7c5f1887fa01f => generated 0 bytes in 564 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 11:59:54.493016 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3f8d36-05cd-4412-babe-5e47df44c225 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:59:54.720795 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0d1817-4cff-4b04-8471-9a09d241369a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:59:54.721733 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0d1817-4cff-4b04-8471-9a09d241369a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:54.722733 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0d1817-4cff-4b04-8471-9a09d241369a admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:54.723202 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0d1817-4cff-4b04-8471-9a09d241369a admin admin] PATH_INFO: `/v3/users/52485eaed13c41b49e16a503b557e138` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:54.727496 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0d1817-4cff-4b04-8471-9a09d241369a admin admin] RBAC: Authorizing `identity:delete_user(user_id=52485eaed13c41b49e16a503b557e138)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:59:54.730766 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0d1817-4cff-4b04-8471-9a09d241369a admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:59:54.841942 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-af0d1817-4cff-4b04-8471-9a09d241369a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52485eaed13c41b49e16a503b557e138'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 11:59:54.857613 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:59:54.874100 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-af0d1817-4cff-4b04-8471-9a09d241369a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52485eaed13c41b49e16a503b557e138'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 11:59:54.876152 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-af0d1817-4cff-4b04-8471-9a09d241369a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52485eaed13c41b49e16a503b557e138'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 11:59:54.898028 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-af0d1817-4cff-4b04-8471-9a09d241369a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52485eaed13c41b49e16a503b557e138'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 11:59:54.898965 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-af0d1817-4cff-4b04-8471-9a09d241369a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52485eaed13c41b49e16a503b557e138'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 11:59:54.939976 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 676/1349] 10.222.0.59 () {58 vars in 1224 bytes} [Tue Oct 11 11:59:54 2022] DELETE /identity/v3/users/52485eaed13c41b49e16a503b557e138 => generated 0 bytes in 731 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 11:59:55.385929 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f45b7bf-5b7e-4fa9-82cd-ff8c7c86f178 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:59:55.388347 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-7f45b7bf-5b7e-4fa9-82cd-ff8c7c86f178 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:59:55.429908 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f45b7bf-5b7e-4fa9-82cd-ff8c7c86f178 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:59:55.604763 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d60c11-179c-4c31-8ff2-b4b99732baf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:59:55.605719 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d60c11-179c-4c31-8ff2-b4b99732baf2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:55.606278 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d60c11-179c-4c31-8ff2-b4b99732baf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:55.606966 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d60c11-179c-4c31-8ff2-b4b99732baf2 admin admin] PATH_INFO: `/v3/projects/3dac33acae7c4016b680cf0bd68ecfee` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:55.608500 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d60c11-179c-4c31-8ff2-b4b99732baf2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3dac33acae7c4016b680cf0bd68ecfee)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:59:55.611192 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d60c11-179c-4c31-8ff2-b4b99732baf2 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:59:55.756660 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-f8d60c11-179c-4c31-8ff2-b4b99732baf2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3dac33acae7c4016b680cf0bd68ecfee', 'request_id': 'req-f8d60c11-179c-4c31-8ff2-b4b99732baf2'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 11:59:55.780357 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-f8d60c11-179c-4c31-8ff2-b4b99732baf2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3dac33acae7c4016b680cf0bd68ecfee', 'request_id': 'req-f8d60c11-179c-4c31-8ff2-b4b99732baf2'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 11:59:55.800059 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-f8d60c11-179c-4c31-8ff2-b4b99732baf2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3dac33acae7c4016b680cf0bd68ecfee', 'request_id': 'req-f8d60c11-179c-4c31-8ff2-b4b99732baf2'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 11:59:55.870310 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-f8d60c11-179c-4c31-8ff2-b4b99732baf2 admin admin] The token cache is being invalidate because project 3dac33acae7c4016b680cf0bd68ecfee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 11:59:55.871623 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-f8d60c11-179c-4c31-8ff2-b4b99732baf2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 11:59:55.872808 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-f8d60c11-179c-4c31-8ff2-b4b99732baf2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 11:59:55.878289 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 674/1350] 10.222.0.59 () {58 vars in 1233 bytes} [Tue Oct 11 11:59:55 2022] DELETE /identity/v3/projects/3dac33acae7c4016b680cf0bd68ecfee => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 11:59:55.884359 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc45a1b-1580-4a07-ad8c-4626afe7659b None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:55.886186 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc45a1b-1580-4a07-ad8c-4626afe7659b None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:55.887857 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc45a1b-1580-4a07-ad8c-4626afe7659b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:55.970523 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-7fc45a1b-1580-4a07-ad8c-4626afe7659b None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:59:55.978055 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-7fc45a1b-1580-4a07-ad8c-4626afe7659b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:59:56.100476 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d60c11-179c-4c31-8ff2-b4b99732baf2 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:59:56.102114 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-f8d60c11-179c-4c31-8ff2-b4b99732baf2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:59:56.244956 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 677/1351] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 11:59:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 362 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 11:59:56.258822 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeaaa69-553a-4596-a743-3d3281742d32 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:56.259376 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeaaa69-553a-4596-a743-3d3281742d32 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:56.260956 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeaaa69-553a-4596-a743-3d3281742d32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:56.311916 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-0eeaaa69-553a-4596-a743-3d3281742d32 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:59:56.328775 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-0eeaaa69-553a-4596-a743-3d3281742d32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:59:56.329614 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d60c11-179c-4c31-8ff2-b4b99732baf2 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:59:56.336584 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7bc094-abd4-4c4d-8b4a-84fc199aceb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:59:56.337502 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7bc094-abd4-4c4d-8b4a-84fc199aceb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:56.338038 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7bc094-abd4-4c4d-8b4a-84fc199aceb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:56.338702 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7bc094-abd4-4c4d-8b4a-84fc199aceb6 admin admin] PATH_INFO: `/v3/projects/5a87190cffa1499f901d96e37abf7190` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:56.341256 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7bc094-abd4-4c4d-8b4a-84fc199aceb6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a87190cffa1499f901d96e37abf7190)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:59:56.343854 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7bc094-abd4-4c4d-8b4a-84fc199aceb6 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:59:56.355446 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:59:56.398738 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 678/1352] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 11:59:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 11:59:56.400450 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eeaaa69-553a-4596-a743-3d3281742d32 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:59:56.443314 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eeaaa69-553a-4596-a743-3d3281742d32 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:59:56.470092 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-7d7bc094-abd4-4c4d-8b4a-84fc199aceb6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a87190cffa1499f901d96e37abf7190', 'request_id': 'req-7d7bc094-abd4-4c4d-8b4a-84fc199aceb6'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 11:59:56.485738 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-7d7bc094-abd4-4c4d-8b4a-84fc199aceb6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a87190cffa1499f901d96e37abf7190', 'request_id': 'req-7d7bc094-abd4-4c4d-8b4a-84fc199aceb6'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 11:59:56.503408 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-7d7bc094-abd4-4c4d-8b4a-84fc199aceb6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a87190cffa1499f901d96e37abf7190', 'request_id': 'req-7d7bc094-abd4-4c4d-8b4a-84fc199aceb6'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 11:59:56.535301 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-7d7bc094-abd4-4c4d-8b4a-84fc199aceb6 admin admin] The token cache is being invalidate because project 5a87190cffa1499f901d96e37abf7190 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 11:59:56.535709 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-7d7bc094-abd4-4c4d-8b4a-84fc199aceb6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 11:59:56.536984 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-7d7bc094-abd4-4c4d-8b4a-84fc199aceb6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 11:59:56.545704 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 675/1353] 10.222.0.59 () {58 vars in 1233 bytes} [Tue Oct 11 11:59:56 2022] DELETE /identity/v3/projects/5a87190cffa1499f901d96e37abf7190 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 11:59:56.547498 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bc7dcd-970c-441f-b7e2-302cefe56294 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:56.550016 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bc7dcd-970c-441f-b7e2-302cefe56294 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:56.550402 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bc7dcd-970c-441f-b7e2-302cefe56294 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:56.601966 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-a4bc7dcd-970c-441f-b7e2-302cefe56294 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:59:56.603400 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-a4bc7dcd-970c-441f-b7e2-302cefe56294 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:59:56.742922 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8f59cc-e2d8-4f35-a4e0-5f1b50b23456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:59:56.743837 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8f59cc-e2d8-4f35-a4e0-5f1b50b23456 admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:56.749798 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8f59cc-e2d8-4f35-a4e0-5f1b50b23456 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:56.750357 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8f59cc-e2d8-4f35-a4e0-5f1b50b23456 admin admin] PATH_INFO: `/v3/domains` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:56.752054 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8f59cc-e2d8-4f35-a4e0-5f1b50b23456 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:59:56.752572 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8f59cc-e2d8-4f35-a4e0-5f1b50b23456 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:59:56.759016 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8f59cc-e2d8-4f35-a4e0-5f1b50b23456 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:59:56.808740 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 676/1354] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 11:59:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 11:59:56.812774 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c766bdc9-dc9c-4394-8501-94687ea40b10 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:56.813306 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c766bdc9-dc9c-4394-8501-94687ea40b10 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:56.813764 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c766bdc9-dc9c-4394-8501-94687ea40b10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:56.820043 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 679/1355] 10.222.0.59 () {58 vars in 1153 bytes} [Tue Oct 11 11:59:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 420 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:59:56.822143 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8f59cc-e2d8-4f35-a4e0-5f1b50b23456 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:59:56.834991 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-8e8f59cc-e2d8-4f35-a4e0-5f1b50b23456 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:59:56.866251 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8f59cc-e2d8-4f35-a4e0-5f1b50b23456 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:59:56.876417 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-c766bdc9-dc9c-4394-8501-94687ea40b10 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:59:56.878322 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-c766bdc9-dc9c-4394-8501-94687ea40b10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:59:56.889570 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dc4bea-332d-4611-9bdc-253c51bbefe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:59:56.890640 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dc4bea-332d-4611-9bdc-253c51bbefe9 admin admin] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:56.891080 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dc4bea-332d-4611-9bdc-253c51bbefe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:56.891590 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dc4bea-332d-4611-9bdc-253c51bbefe9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:56.892591 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dc4bea-332d-4611-9bdc-253c51bbefe9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:59:56.896239 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dc4bea-332d-4611-9bdc-253c51bbefe9 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:59:56.925894 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 677/1356] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 11:59:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 11:59:56.927744 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c766bdc9-dc9c-4394-8501-94687ea40b10 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:59:56.981021 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c766bdc9-dc9c-4394-8501-94687ea40b10 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:59:57.010625 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc615778-cbea-4bf7-bbbb-0ddf93bbee54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:59:57.011627 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc615778-cbea-4bf7-bbbb-0ddf93bbee54 admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:57.012121 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc615778-cbea-4bf7-bbbb-0ddf93bbee54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:57.012717 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc615778-cbea-4bf7-bbbb-0ddf93bbee54 admin admin] PATH_INFO: `/v3/domains` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:57.035431 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc615778-cbea-4bf7-bbbb-0ddf93bbee54 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:59:57.036158 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc615778-cbea-4bf7-bbbb-0ddf93bbee54 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:59:57.044753 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc615778-cbea-4bf7-bbbb-0ddf93bbee54 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:59:57.064913 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 680/1357] 10.222.0.59 () {60 vars in 1153 bytes} [Tue Oct 11 11:59:56 2022] POST /identity/v3/projects => generated 383 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 11:59:57.069668 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dc4bea-332d-4611-9bdc-253c51bbefe9 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:59:57.090089 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 678/1358] 10.222.0.59 () {58 vars in 1153 bytes} [Tue Oct 11 11:59:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:59:57.096800 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc615778-cbea-4bf7-bbbb-0ddf93bbee54 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:59:57.125201 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc615778-cbea-4bf7-bbbb-0ddf93bbee54 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:59:57.209140 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:59:57.272811 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dc4bea-332d-4611-9bdc-253c51bbefe9 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:59:57.300556 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8a30bd-cb3f-4fe8-8cea-db9cc7e0a0c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:59:57.301656 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8a30bd-cb3f-4fe8-8cea-db9cc7e0a0c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:57.302113 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8a30bd-cb3f-4fe8-8cea-db9cc7e0a0c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:57.302532 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8a30bd-cb3f-4fe8-8cea-db9cc7e0a0c3 admin admin] PATH_INFO: `/v3/users` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:57.304206 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8a30bd-cb3f-4fe8-8cea-db9cc7e0a0c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:59:57.307841 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8a30bd-cb3f-4fe8-8cea-db9cc7e0a0c3 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:59:57.369210 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ee2053-c46c-4ae8-9ef8-b9cfde9739b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:59:57.370193 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ee2053-c46c-4ae8-9ef8-b9cfde9739b1 admin admin] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:57.370836 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ee2053-c46c-4ae8-9ef8-b9cfde9739b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:57.371243 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ee2053-c46c-4ae8-9ef8-b9cfde9739b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:57.372540 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ee2053-c46c-4ae8-9ef8-b9cfde9739b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:59:57.376974 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ee2053-c46c-4ae8-9ef8-b9cfde9739b1 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:59:57.401546 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 681/1359] 10.222.0.59 () {60 vars in 1144 bytes} [Tue Oct 11 11:59:57 2022] POST /identity/v3/users => generated 292 bytes in 333 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 11:59:57.406481 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8a30bd-cb3f-4fe8-8cea-db9cc7e0a0c3 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:59:57.447175 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8a30bd-cb3f-4fe8-8cea-db9cc7e0a0c3 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:59:57.465014 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3464e484-4ac1-4faf-a89a-f77d22a98182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:59:57.466142 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3464e484-4ac1-4faf-a89a-f77d22a98182 admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:57.468444 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3464e484-4ac1-4faf-a89a-f77d22a98182 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:57.469015 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3464e484-4ac1-4faf-a89a-f77d22a98182 admin admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:57.471166 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3464e484-4ac1-4faf-a89a-f77d22a98182 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:59:57.476655 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3464e484-4ac1-4faf-a89a-f77d22a98182 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:59:57.487507 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 679/1360] 10.222.0.59 () {60 vars in 1153 bytes} [Tue Oct 11 11:59:57 2022] POST /identity/v3/projects => generated 381 bytes in 392 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 11:59:57.491094 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ee2053-c46c-4ae8-9ef8-b9cfde9739b1 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:59:57.503889 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:59:57.537809 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 682/1361] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 11:59:57 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 11:59:57.540310 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3464e484-4ac1-4faf-a89a-f77d22a98182 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:59:57.607485 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3464e484-4ac1-4faf-a89a-f77d22a98182 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:59:57.747033 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ee2053-c46c-4ae8-9ef8-b9cfde9739b1 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:59:57.764971 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ae8753-4193-47e6-afb7-4a6049a09dc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:59:57.765616 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ae8753-4193-47e6-afb7-4a6049a09dc5 admin admin] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:57.765933 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ae8753-4193-47e6-afb7-4a6049a09dc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:57.767212 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ae8753-4193-47e6-afb7-4a6049a09dc5 admin admin] PATH_INFO: `/v3/users` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:57.767983 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ae8753-4193-47e6-afb7-4a6049a09dc5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:59:57.772349 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ae8753-4193-47e6-afb7-4a6049a09dc5 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:59:57.829546 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd314692-2229-4ee2-a323-4780ddadd635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:59:57.830847 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd314692-2229-4ee2-a323-4780ddadd635 admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:57.831470 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd314692-2229-4ee2-a323-4780ddadd635 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:57.832113 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd314692-2229-4ee2-a323-4780ddadd635 admin admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:57.833240 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 680/1362] 10.222.0.59 () {60 vars in 1144 bytes} [Tue Oct 11 11:59:57 2022] POST /identity/v3/users => generated 291 bytes in 345 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 11:59:57.833744 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd314692-2229-4ee2-a323-4780ddadd635 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:59:57.834440 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ae8753-4193-47e6-afb7-4a6049a09dc5 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:59:57.837276 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-00ae8753-4193-47e6-afb7-4a6049a09dc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:59:57.840536 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd314692-2229-4ee2-a323-4780ddadd635 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:59:57.858160 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ae8753-4193-47e6-afb7-4a6049a09dc5 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:59:57.864568 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c000bd4c-9773-45b5-b545-95c1a22c059f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:59:57.865256 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c000bd4c-9773-45b5-b545-95c1a22c059f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:57.865599 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c000bd4c-9773-45b5-b545-95c1a22c059f admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:57.865875 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c000bd4c-9773-45b5-b545-95c1a22c059f admin admin] PATH_INFO: `/v3/projects/74f3246047bd45cda1eaa9c0cd8137da` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:57.867451 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c000bd4c-9773-45b5-b545-95c1a22c059f admin admin] RBAC: Authorizing `identity:delete_project(project_id=74f3246047bd45cda1eaa9c0cd8137da)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:59:57.869768 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c000bd4c-9773-45b5-b545-95c1a22c059f admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:59:57.888985 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 683/1363] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 11:59:57 2022] GET /identity/v3/roles => generated 1420 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 11 11:59:57.890919 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd314692-2229-4ee2-a323-4780ddadd635 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:59:57.932360 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd314692-2229-4ee2-a323-4780ddadd635 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:59:57.946763 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dbc877-16ab-4b3f-ac4e-03a1b90ef7a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:59:57.947629 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dbc877-16ab-4b3f-ac4e-03a1b90ef7a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:57.947918 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dbc877-16ab-4b3f-ac4e-03a1b90ef7a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:57.948930 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dbc877-16ab-4b3f-ac4e-03a1b90ef7a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:57.949628 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dbc877-16ab-4b3f-ac4e-03a1b90ef7a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:59:57.951388 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dbc877-16ab-4b3f-ac4e-03a1b90ef7a7 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:59:57.976834 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-c000bd4c-9773-45b5-b545-95c1a22c059f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74f3246047bd45cda1eaa9c0cd8137da', 'request_id': 'req-c000bd4c-9773-45b5-b545-95c1a22c059f'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 11:59:57.987376 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-c000bd4c-9773-45b5-b545-95c1a22c059f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74f3246047bd45cda1eaa9c0cd8137da', 'request_id': 'req-c000bd4c-9773-45b5-b545-95c1a22c059f'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 11:59:57.995955 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 684/1364] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 11:59:57 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 11:59:57.997295 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dbc877-16ab-4b3f-ac4e-03a1b90ef7a7 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:59:58.003023 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-c000bd4c-9773-45b5-b545-95c1a22c059f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74f3246047bd45cda1eaa9c0cd8137da', 'request_id': 'req-c000bd4c-9773-45b5-b545-95c1a22c059f'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 11:59:58.020399 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dbc877-16ab-4b3f-ac4e-03a1b90ef7a7 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:59:58.057349 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-c000bd4c-9773-45b5-b545-95c1a22c059f admin admin] The token cache is being invalidate because project 74f3246047bd45cda1eaa9c0cd8137da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 11:59:58.058277 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-c000bd4c-9773-45b5-b545-95c1a22c059f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 11:59:58.060089 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-c000bd4c-9773-45b5-b545-95c1a22c059f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 11:59:58.064809 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 681/1365] 10.222.0.59 () {58 vars in 1233 bytes} [Tue Oct 11 11:59:57 2022] DELETE /identity/v3/projects/74f3246047bd45cda1eaa9c0cd8137da => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 11:59:58.074677 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c000bd4c-9773-45b5-b545-95c1a22c059f admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:59:58.078648 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-c000bd4c-9773-45b5-b545-95c1a22c059f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:59:58.137624 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e45755-14c6-4f0b-b920-42cc375fa41d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:59:58.165019 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e45755-14c6-4f0b-b920-42cc375fa41d admin admin] REQUEST_METHOD: `PUT` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:58.168807 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e45755-14c6-4f0b-b920-42cc375fa41d admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:58.169386 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e45755-14c6-4f0b-b920-42cc375fa41d admin admin] PATH_INFO: `/v3/projects/c77e600b1b3746aeb4468fb01886b74d/users/2b616e9828314e95937990d0a9278c60/roles/de2d2bd26fff4748a79990f97a107164` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:58.211366 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c000bd4c-9773-45b5-b545-95c1a22c059f admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:59:58.218822 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e959fa67-7476-4b42-963a-7436ecb8c591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:59:58.219515 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e959fa67-7476-4b42-963a-7436ecb8c591 admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:58.219965 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e959fa67-7476-4b42-963a-7436ecb8c591 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:58.220402 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e959fa67-7476-4b42-963a-7436ecb8c591 admin admin] PATH_INFO: `/v3/roles` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:58.221702 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e959fa67-7476-4b42-963a-7436ecb8c591 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:59:58.224559 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e959fa67-7476-4b42-963a-7436ecb8c591 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:59:58.234790 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e45755-14c6-4f0b-b920-42cc375fa41d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c77e600b1b3746aeb4468fb01886b74d, user_id=2b616e9828314e95937990d0a9278c60, role_id=de2d2bd26fff4748a79990f97a107164)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:59:58.239679 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e45755-14c6-4f0b-b920-42cc375fa41d admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:59:58.242218 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 682/1366] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 11:59:58 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 11:59:58.245576 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937df795-ac73-4af9-87fa-e6e9580c38d2 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:58.245943 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937df795-ac73-4af9-87fa-e6e9580c38d2 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:58.246678 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937df795-ac73-4af9-87fa-e6e9580c38d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:58.266282 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 685/1367] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 11:59:57 2022] PUT /identity/v3/projects/c77e600b1b3746aeb4468fb01886b74d/users/2b616e9828314e95937990d0a9278c60/roles/de2d2bd26fff4748a79990f97a107164 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 11:59:58.268839 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e45755-14c6-4f0b-b920-42cc375fa41d admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:59:58.300234 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-937df795-ac73-4af9-87fa-e6e9580c38d2 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:59:58.301934 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-937df795-ac73-4af9-87fa-e6e9580c38d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:59:58.317281 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:59:58.343368 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:59:58.420128 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e45755-14c6-4f0b-b920-42cc375fa41d admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:59:58.433157 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71aec21-db98-45a1-b8b2-05cf0cc24c37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:59:58.434614 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71aec21-db98-45a1-b8b2-05cf0cc24c37 admin admin] REQUEST_METHOD: `PUT` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:58.436289 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71aec21-db98-45a1-b8b2-05cf0cc24c37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:58.437537 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71aec21-db98-45a1-b8b2-05cf0cc24c37 admin admin] PATH_INFO: `/v3/projects/6ec4547f0ca04016a113f272755e718a/users/4fe1a7003a7f402c9f77cc75f161d4d5/roles/de2d2bd26fff4748a79990f97a107164` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:58.467879 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 683/1368] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 11:59:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 11:59:58.467879 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c62c6c-4541-4b5d-aed7-4604ed596589 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:58.467879 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c62c6c-4541-4b5d-aed7-4604ed596589 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:58.467879 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c62c6c-4541-4b5d-aed7-4604ed596589 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:58.472570 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71aec21-db98-45a1-b8b2-05cf0cc24c37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ec4547f0ca04016a113f272755e718a, user_id=4fe1a7003a7f402c9f77cc75f161d4d5, role_id=de2d2bd26fff4748a79990f97a107164)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:59:58.475675 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71aec21-db98-45a1-b8b2-05cf0cc24c37 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:59:58.522139 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 686/1369] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 11:59:58 2022] PUT /identity/v3/projects/6ec4547f0ca04016a113f272755e718a/users/4fe1a7003a7f402c9f77cc75f161d4d5/roles/de2d2bd26fff4748a79990f97a107164 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 11:59:58.523996 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7995891-efdf-4898-839f-794710f9e238 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:58.524456 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7995891-efdf-4898-839f-794710f9e238 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:58.524993 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7995891-efdf-4898-839f-794710f9e238 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:58.570357 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-39c62c6c-4541-4b5d-aed7-4604ed596589 None None] MFA Rules not processed for user `2b616e9828314e95937990d0a9278c60`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:59:58.570357 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-39c62c6c-4541-4b5d-aed7-4604ed596589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:59:58.661480 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-f7995891-efdf-4898-839f-794710f9e238 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:59:58.668030 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-f7995891-efdf-4898-839f-794710f9e238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:59:58.816766 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 684/1370] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 11:59:58 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 350 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 11:59:58.816766 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde9544c-92be-4b01-a36a-1b0b5db5aad7 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:58.816766 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde9544c-92be-4b01-a36a-1b0b5db5aad7 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:58.816766 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde9544c-92be-4b01-a36a-1b0b5db5aad7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:58.925204 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-cde9544c-92be-4b01-a36a-1b0b5db5aad7 None None] MFA Rules not processed for user `4fe1a7003a7f402c9f77cc75f161d4d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 11:59:58.925204 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-cde9544c-92be-4b01-a36a-1b0b5db5aad7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:59:59.040458 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 687/1371] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 11:59:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 518 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 11:59:59.044553 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7995891-efdf-4898-839f-794710f9e238 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:59:59.048840 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-f7995891-efdf-4898-839f-794710f9e238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 11:59:59.071694 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7995891-efdf-4898-839f-794710f9e238 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:59:59.096389 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad7fe25-8b61-46d4-9b56-335a33a77e68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:59:59.097371 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad7fe25-8b61-46d4-9b56-335a33a77e68 admin admin] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:59.097810 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad7fe25-8b61-46d4-9b56-335a33a77e68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:59.098300 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad7fe25-8b61-46d4-9b56-335a33a77e68 admin admin] PATH_INFO: `/v3/projects` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:59.099237 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad7fe25-8b61-46d4-9b56-335a33a77e68 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:59:59.102317 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad7fe25-8b61-46d4-9b56-335a33a77e68 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:59:59.110536 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 685/1372] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 11:59:58 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 11:59:59.111883 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde9544c-92be-4b01-a36a-1b0b5db5aad7 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:59:59.142698 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde9544c-92be-4b01-a36a-1b0b5db5aad7 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:59:59.152185 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1f08ac-210f-453c-b002-b73285f34d8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:59:59.152840 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1f08ac-210f-453c-b002-b73285f34d8a admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:59.153518 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1f08ac-210f-453c-b002-b73285f34d8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:59.153776 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1f08ac-210f-453c-b002-b73285f34d8a admin admin] PATH_INFO: `/v3/domains` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:59.154452 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1f08ac-210f-453c-b002-b73285f34d8a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 11:59:59.154797 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1f08ac-210f-453c-b002-b73285f34d8a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:59:59.156480 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1f08ac-210f-453c-b002-b73285f34d8a admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:59:59.173885 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 688/1373] 10.222.0.59 () {60 vars in 1153 bytes} [Tue Oct 11 11:59:59 2022] POST /identity/v3/projects => generated 387 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 11:59:59.174817 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad7fe25-8b61-46d4-9b56-335a33a77e68 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:59:59.187329 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 686/1374] 10.222.0.59 () {58 vars in 1153 bytes} [Tue Oct 11 11:59:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 11:59:59.188351 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1f08ac-210f-453c-b002-b73285f34d8a admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:59:59.215471 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1f08ac-210f-453c-b002-b73285f34d8a admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:59:59.244226 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:59:59.308911 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad7fe25-8b61-46d4-9b56-335a33a77e68 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:59:59.321700 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27632914-1b3c-4c6f-ba60-a7d8b157f818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:59:59.322911 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27632914-1b3c-4c6f-ba60-a7d8b157f818 admin admin] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:59.323193 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27632914-1b3c-4c6f-ba60-a7d8b157f818 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:59.323474 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27632914-1b3c-4c6f-ba60-a7d8b157f818 admin admin] PATH_INFO: `/v3/projects` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:59.324143 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27632914-1b3c-4c6f-ba60-a7d8b157f818 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:59:59.327426 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27632914-1b3c-4c6f-ba60-a7d8b157f818 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:59:59.342113 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d63ca0-1963-4dac-84e3-19e9639615ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:59:59.342801 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d63ca0-1963-4dac-84e3-19e9639615ef admin admin] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:59.343090 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d63ca0-1963-4dac-84e3-19e9639615ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:59.343369 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d63ca0-1963-4dac-84e3-19e9639615ef admin admin] PATH_INFO: `/v3/users` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:59.344055 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d63ca0-1963-4dac-84e3-19e9639615ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:59:59.347464 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d63ca0-1963-4dac-84e3-19e9639615ef admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:59:59.407998 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 689/1375] 10.222.0.59 () {60 vars in 1153 bytes} [Tue Oct 11 11:59:59 2022] POST /identity/v3/projects => generated 381 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 11:59:59.410291 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27632914-1b3c-4c6f-ba60-a7d8b157f818 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:59:59.416234 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 687/1376] 10.222.0.59 () {60 vars in 1144 bytes} [Tue Oct 11 11:59:59 2022] POST /identity/v3/users => generated 294 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 11:59:59.417538 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d63ca0-1963-4dac-84e3-19e9639615ef admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:59:59.430257 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d63ca0-1963-4dac-84e3-19e9639615ef admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:59:59.447218 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 11:59:59.562893 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27632914-1b3c-4c6f-ba60-a7d8b157f818 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:59:59.568608 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f5d686-8883-4b00-b351-2507b3be275f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:59:59.569266 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f5d686-8883-4b00-b351-2507b3be275f admin admin] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:59.569550 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f5d686-8883-4b00-b351-2507b3be275f admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:59.569833 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f5d686-8883-4b00-b351-2507b3be275f admin admin] PATH_INFO: `/v3/projects` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:59.570495 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f5d686-8883-4b00-b351-2507b3be275f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:59:59.572571 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f5d686-8883-4b00-b351-2507b3be275f admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:59:59.599811 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3236aa46-f64c-416f-9856-166cec6f0d4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:59:59.601006 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3236aa46-f64c-416f-9856-166cec6f0d4d admin admin] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:59.603060 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3236aa46-f64c-416f-9856-166cec6f0d4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:59.604746 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3236aa46-f64c-416f-9856-166cec6f0d4d admin admin] PATH_INFO: `/v3/users` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:59.607249 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3236aa46-f64c-416f-9856-166cec6f0d4d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:59:59.609676 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3236aa46-f64c-416f-9856-166cec6f0d4d admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:59:59.627630 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 690/1377] 10.222.0.59 () {60 vars in 1153 bytes} [Tue Oct 11 11:59:59 2022] POST /identity/v3/projects => generated 405 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 11:59:59.629141 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f5d686-8883-4b00-b351-2507b3be275f admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:59:59.656268 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 688/1378] 10.222.0.59 () {60 vars in 1144 bytes} [Tue Oct 11 11:59:59 2022] POST /identity/v3/users => generated 297 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 11:59:59.657435 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3236aa46-f64c-416f-9856-166cec6f0d4d admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:59:59.674345 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3236aa46-f64c-416f-9856-166cec6f0d4d admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:59:59.822543 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f5d686-8883-4b00-b351-2507b3be275f admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:59:59.828170 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d974fd79-bbd4-4f4b-877b-6afefea5c410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:59:59.828846 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d974fd79-bbd4-4f4b-877b-6afefea5c410 admin admin] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:59.829115 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d974fd79-bbd4-4f4b-877b-6afefea5c410 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:59.829386 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d974fd79-bbd4-4f4b-877b-6afefea5c410 admin admin] PATH_INFO: `/v3/users` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:59.831207 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d974fd79-bbd4-4f4b-877b-6afefea5c410 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:59:59.834283 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d974fd79-bbd4-4f4b-877b-6afefea5c410 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:59:59.839193 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb5bdcf-b18d-4c1a-a000-7f89149bc8df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:59:59.839830 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb5bdcf-b18d-4c1a-a000-7f89149bc8df admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:59.840191 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb5bdcf-b18d-4c1a-a000-7f89149bc8df admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:59.840387 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb5bdcf-b18d-4c1a-a000-7f89149bc8df admin admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:59.841031 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb5bdcf-b18d-4c1a-a000-7f89149bc8df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:59:59.842747 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb5bdcf-b18d-4c1a-a000-7f89149bc8df admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:59:59.861739 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 691/1379] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 11:59:59 2022] GET /identity/v3/roles => generated 1420 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 11:59:59.863263 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb5bdcf-b18d-4c1a-a000-7f89149bc8df admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:59:59.886427 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb5bdcf-b18d-4c1a-a000-7f89149bc8df admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:59:59.894929 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 689/1380] 10.222.0.59 () {60 vars in 1144 bytes} [Tue Oct 11 11:59:59 2022] POST /identity/v3/users => generated 303 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 11:59:59.894929 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d974fd79-bbd4-4f4b-877b-6afefea5c410 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:59:59.904782 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444183b5-d1a3-404f-8009-c7850f86a030 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:59:59.906244 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444183b5-d1a3-404f-8009-c7850f86a030 admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:59.906763 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444183b5-d1a3-404f-8009-c7850f86a030 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:59.907115 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444183b5-d1a3-404f-8009-c7850f86a030 admin admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:59.907902 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444183b5-d1a3-404f-8009-c7850f86a030 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:59:59.910889 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444183b5-d1a3-404f-8009-c7850f86a030 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:59:59.926611 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d974fd79-bbd4-4f4b-877b-6afefea5c410 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 11:59:59.939047 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41acd08d-deee-4ea0-9181-83aadf5e2940 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 11:59:59.941125 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41acd08d-deee-4ea0-9181-83aadf5e2940 admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 11:59:59.941817 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41acd08d-deee-4ea0-9181-83aadf5e2940 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 11:59:59.942280 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41acd08d-deee-4ea0-9181-83aadf5e2940 admin admin] PATH_INFO: `/v3/roles` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 11:59:59.943331 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41acd08d-deee-4ea0-9181-83aadf5e2940 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 11:59:59.946939 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41acd08d-deee-4ea0-9181-83aadf5e2940 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 11:59:59.983200 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 690/1382] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 11:59:59 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 11:59:59.983481 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 692/1382] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 11:59:59 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 11:59:59.983642 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41acd08d-deee-4ea0-9181-83aadf5e2940 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 11:59:59.983842 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444183b5-d1a3-404f-8009-c7850f86a030 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:00.005326 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41acd08d-deee-4ea0-9181-83aadf5e2940 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:00.014610 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444183b5-d1a3-404f-8009-c7850f86a030 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:00.023541 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53db8aab-f59c-4eed-a8f7-182d26303ebf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:00.024772 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53db8aab-f59c-4eed-a8f7-182d26303ebf admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:00.025275 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53db8aab-f59c-4eed-a8f7-182d26303ebf admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:00.025777 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53db8aab-f59c-4eed-a8f7-182d26303ebf admin admin] PATH_INFO: `/v3/roles` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:00.026715 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53db8aab-f59c-4eed-a8f7-182d26303ebf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:00.029179 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53db8aab-f59c-4eed-a8f7-182d26303ebf admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:00.033695 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86f5545-0fd1-4ef9-8be4-e74e0bee98dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:00.034655 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86f5545-0fd1-4ef9-8be4-e74e0bee98dc admin admin] REQUEST_METHOD: `PUT` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:00.035119 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86f5545-0fd1-4ef9-8be4-e74e0bee98dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:00.035642 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86f5545-0fd1-4ef9-8be4-e74e0bee98dc admin admin] PATH_INFO: `/v3/projects/327ec774905a459c894e7842ba67776c/users/76d6e00fb4c84488a0cc0846c0e8bdbc/roles/de2d2bd26fff4748a79990f97a107164` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:00.050977 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 691/1383] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 11:59:59 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:00:00.053053 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53db8aab-f59c-4eed-a8f7-182d26303ebf admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:00.076395 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86f5545-0fd1-4ef9-8be4-e74e0bee98dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=327ec774905a459c894e7842ba67776c, user_id=76d6e00fb4c84488a0cc0846c0e8bdbc, role_id=de2d2bd26fff4748a79990f97a107164)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:00.082765 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86f5545-0fd1-4ef9-8be4-e74e0bee98dc admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:00.085484 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:00:00.100217 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53db8aab-f59c-4eed-a8f7-182d26303ebf admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:00.117375 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e78625-2cea-473d-a298-bbd13ba41592 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:00.119295 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e78625-2cea-473d-a298-bbd13ba41592 admin admin] REQUEST_METHOD: `PUT` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:00.128824 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e78625-2cea-473d-a298-bbd13ba41592 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:00.129179 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e78625-2cea-473d-a298-bbd13ba41592 admin admin] PATH_INFO: `/v3/projects/cda9f60831234c0ba6c5d67dc8cfb7b7/users/37f8c72e7ac44e4d97f0e85646539db4/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:00.155400 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e78625-2cea-473d-a298-bbd13ba41592 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cda9f60831234c0ba6c5d67dc8cfb7b7, user_id=37f8c72e7ac44e4d97f0e85646539db4, role_id=5cf1f4bd2fef46ab9c6eff56d52f9f69)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:00.162745 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 693/1384] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 11:59:59 2022] PUT /identity/v3/projects/327ec774905a459c894e7842ba67776c/users/76d6e00fb4c84488a0cc0846c0e8bdbc/roles/de2d2bd26fff4748a79990f97a107164 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:00:00.162745 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86f5545-0fd1-4ef9-8be4-e74e0bee98dc admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:00.163858 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e78625-2cea-473d-a298-bbd13ba41592 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:00.190229 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 692/1385] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 12:00:00 2022] PUT /identity/v3/projects/cda9f60831234c0ba6c5d67dc8cfb7b7/users/37f8c72e7ac44e4d97f0e85646539db4/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:00:00.193245 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0796b43-310a-4f84-8ece-349563389f20 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:00.193858 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0796b43-310a-4f84-8ece-349563389f20 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:00.194626 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0796b43-310a-4f84-8ece-349563389f20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:00.291548 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-e0796b43-310a-4f84-8ece-349563389f20 None None] MFA Rules not processed for user `76d6e00fb4c84488a0cc0846c0e8bdbc`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:00:00.293745 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-e0796b43-310a-4f84-8ece-349563389f20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:00.393353 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86f5545-0fd1-4ef9-8be4-e74e0bee98dc admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:00.409335 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:00:00.472721 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 693/1386] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:00:00 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:00:00.473975 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0796b43-310a-4f84-8ece-349563389f20 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:00.598715 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0796b43-310a-4f84-8ece-349563389f20 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:00.604946 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3feae6-621e-4898-9548-adf7a603cc31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:00.606395 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3feae6-621e-4898-9548-adf7a603cc31 admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:00.606820 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3feae6-621e-4898-9548-adf7a603cc31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:00.607231 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3feae6-621e-4898-9548-adf7a603cc31 admin admin] PATH_INFO: `/v3/roles` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:00.608114 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3feae6-621e-4898-9548-adf7a603cc31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:00.610939 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3feae6-621e-4898-9548-adf7a603cc31 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:00.630499 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 694/1387] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:00:00 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:00:00.631920 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3feae6-621e-4898-9548-adf7a603cc31 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:00.663622 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3feae6-621e-4898-9548-adf7a603cc31 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:00.678867 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea6296d-7de9-4174-aa9c-69ab2f15519b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:00.680403 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae676f1-3cba-4b57-86b5-5774bb52d410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:00.681604 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae676f1-3cba-4b57-86b5-5774bb52d410 admin admin] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:00.682010 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae676f1-3cba-4b57-86b5-5774bb52d410 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:00.682432 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae676f1-3cba-4b57-86b5-5774bb52d410 admin admin] PATH_INFO: `/v3/projects` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:00.684878 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae676f1-3cba-4b57-86b5-5774bb52d410 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:00.687379 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae676f1-3cba-4b57-86b5-5774bb52d410 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:00.691468 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea6296d-7de9-4174-aa9c-69ab2f15519b admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:00.692221 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea6296d-7de9-4174-aa9c-69ab2f15519b admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:00.692725 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea6296d-7de9-4174-aa9c-69ab2f15519b admin admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:00.693919 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea6296d-7de9-4174-aa9c-69ab2f15519b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:00.697167 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea6296d-7de9-4174-aa9c-69ab2f15519b admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:00.777031 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 694/1388] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:00:00 2022] GET /identity/v3/roles => generated 1420 bytes in 620 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:00:00.780667 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea6296d-7de9-4174-aa9c-69ab2f15519b admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:00.785601 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 695/1389] 10.222.0.59 () {60 vars in 1153 bytes} [Tue Oct 11 12:00:00 2022] POST /identity/v3/projects => generated 387 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:00:00.792028 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae676f1-3cba-4b57-86b5-5774bb52d410 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:00.838910 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea6296d-7de9-4174-aa9c-69ab2f15519b admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:00.870231 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:00:00.902198 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae676f1-3cba-4b57-86b5-5774bb52d410 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:00.918054 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71f84ab-3fcf-4b67-a34b-4acfbc3b6610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:00.919043 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71f84ab-3fcf-4b67-a34b-4acfbc3b6610 admin admin] REQUEST_METHOD: `PUT` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:00.920332 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71f84ab-3fcf-4b67-a34b-4acfbc3b6610 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:00.927437 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71f84ab-3fcf-4b67-a34b-4acfbc3b6610 admin admin] PATH_INFO: `/v3/projects/c9aa6aaab31e4dc5b5e7aa6ea8b91b0a/users/2fb546bc272b405699262f07ef584c7f/roles/de2d2bd26fff4748a79990f97a107164` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:01.001191 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71f84ab-3fcf-4b67-a34b-4acfbc3b6610 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9aa6aaab31e4dc5b5e7aa6ea8b91b0a, user_id=2fb546bc272b405699262f07ef584c7f, role_id=de2d2bd26fff4748a79990f97a107164)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:01.015731 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71f84ab-3fcf-4b67-a34b-4acfbc3b6610 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:01.066767 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 696/1390] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 12:00:00 2022] PUT /identity/v3/projects/c9aa6aaab31e4dc5b5e7aa6ea8b91b0a/users/2fb546bc272b405699262f07ef584c7f/roles/de2d2bd26fff4748a79990f97a107164 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:00:01.077602 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71f84ab-3fcf-4b67-a34b-4acfbc3b6610 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:01.129502 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead1cecc-257b-4bf3-ac8a-cac93cff58ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:01.130854 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead1cecc-257b-4bf3-ac8a-cac93cff58ee admin admin] REQUEST_METHOD: `PUT` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:01.147086 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead1cecc-257b-4bf3-ac8a-cac93cff58ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:01.147645 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead1cecc-257b-4bf3-ac8a-cac93cff58ee admin admin] PATH_INFO: `/v3/projects/cda9f60831234c0ba6c5d67dc8cfb7b7/users/37f8c72e7ac44e4d97f0e85646539db4/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:01.156348 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead1cecc-257b-4bf3-ac8a-cac93cff58ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=cda9f60831234c0ba6c5d67dc8cfb7b7, user_id=37f8c72e7ac44e4d97f0e85646539db4, role_id=5cf1f4bd2fef46ab9c6eff56d52f9f69)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:01.162784 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead1cecc-257b-4bf3-ac8a-cac93cff58ee admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:01.215319 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 695/1391] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 12:00:00 2022] PUT /identity/v3/projects/cda9f60831234c0ba6c5d67dc8cfb7b7/users/37f8c72e7ac44e4d97f0e85646539db4/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69 => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:00:01.217877 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a52ae03-faf3-4220-b589-1ced0aa5e24f None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:01.218379 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a52ae03-faf3-4220-b589-1ced0aa5e24f None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:01.233621 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a52ae03-faf3-4220-b589-1ced0aa5e24f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:01.277191 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-0a52ae03-faf3-4220-b589-1ced0aa5e24f None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:00:01.280394 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-0a52ae03-faf3-4220-b589-1ced0aa5e24f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:01.380550 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71f84ab-3fcf-4b67-a34b-4acfbc3b6610 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:01.471503 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 696/1392] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:00:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:00:01.474401 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bf9a70-f1db-4c29-b86e-3b6f6753be45 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:01.474941 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bf9a70-f1db-4c29-b86e-3b6f6753be45 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:01.475435 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bf9a70-f1db-4c29-b86e-3b6f6753be45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:01.545141 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb10efa-c9b5-47d8-ac7d-1d7cad53ac96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:01.546521 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb10efa-c9b5-47d8-ac7d-1d7cad53ac96 admin admin] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:01.547173 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb10efa-c9b5-47d8-ac7d-1d7cad53ac96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:01.548140 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb10efa-c9b5-47d8-ac7d-1d7cad53ac96 admin admin] PATH_INFO: `/v3/users` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:01.549503 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb10efa-c9b5-47d8-ac7d-1d7cad53ac96 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:01.560201 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-89bf9a70-f1db-4c29-b86e-3b6f6753be45 None None] MFA Rules not processed for user `37f8c72e7ac44e4d97f0e85646539db4`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:00:01.562111 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb10efa-c9b5-47d8-ac7d-1d7cad53ac96 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:01.562357 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-89bf9a70-f1db-4c29-b86e-3b6f6753be45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:01.634306 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 697/1393] 10.222.0.59 () {60 vars in 1144 bytes} [Tue Oct 11 12:00:01 2022] POST /identity/v3/users => generated 300 bytes in 566 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:00:01.640912 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb10efa-c9b5-47d8-ac7d-1d7cad53ac96 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:01.646770 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-7fb10efa-c9b5-47d8-ac7d-1d7cad53ac96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:01.682100 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:00:01.796130 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 697/1394] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:00:01 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 319 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:00:01.797807 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bf9a70-f1db-4c29-b86e-3b6f6753be45 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:01.844410 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bf9a70-f1db-4c29-b86e-3b6f6753be45 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:01.865192 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6ece80-a3b9-4677-a98c-ad332b3dcd38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:01.869313 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6ece80-a3b9-4677-a98c-ad332b3dcd38 admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:01.869876 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6ece80-a3b9-4677-a98c-ad332b3dcd38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:01.870805 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6ece80-a3b9-4677-a98c-ad332b3dcd38 admin admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:01.877108 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6ece80-a3b9-4677-a98c-ad332b3dcd38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:01.886035 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb10efa-c9b5-47d8-ac7d-1d7cad53ac96 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:01.887773 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6ece80-a3b9-4677-a98c-ad332b3dcd38 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:01.915077 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8effa96-819f-4876-be74-24b9edfe7675 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e563fe174db4cc19ae6bf8162cc3d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:01.916181 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8effa96-819f-4876-be74-24b9edfe7675 service neutron] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:01.916686 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8effa96-819f-4876-be74-24b9edfe7675 service neutron] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:01.917170 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8effa96-819f-4876-be74-24b9edfe7675 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:01.950684 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 698/1395] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:00:01 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:00:01.950684 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6ece80-a3b9-4677-a98c-ad332b3dcd38 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:01.959374 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8effa96-819f-4876-be74-24b9edfe7675 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:01.961496 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8effa96-819f-4876-be74-24b9edfe7675 service neutron] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:01.975805 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-4c6ece80-a3b9-4677-a98c-ad332b3dcd38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:01.996042 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 698/1396] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:00:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:00:02.004104 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8effa96-819f-4876-be74-24b9edfe7675 service neutron] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:02.039430 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8effa96-819f-4876-be74-24b9edfe7675 service neutron] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:02.059659 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b711d2e-c3ee-4a03-afff-7c88e973350f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:02.060956 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b711d2e-c3ee-4a03-afff-7c88e973350f admin admin] REQUEST_METHOD: `PUT` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:02.061308 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b711d2e-c3ee-4a03-afff-7c88e973350f admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:02.061814 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b711d2e-c3ee-4a03-afff-7c88e973350f admin admin] PATH_INFO: `/v3/projects/54ad85255745456a98daf8c94b8a1c25/users/d5ade4db7fe943b2b7d6b33b7c7b5872/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:02.074542 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:00:02.113737 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b711d2e-c3ee-4a03-afff-7c88e973350f admin admin] RBAC: Authorizing `identity:create_grant(project_id=54ad85255745456a98daf8c94b8a1c25, user_id=d5ade4db7fe943b2b7d6b33b7c7b5872, role_id=5cf1f4bd2fef46ab9c6eff56d52f9f69)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:02.116595 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b711d2e-c3ee-4a03-afff-7c88e973350f admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:02.145675 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 699/1397] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 12:00:01 2022] PUT /identity/v3/projects/54ad85255745456a98daf8c94b8a1c25/users/d5ade4db7fe943b2b7d6b33b7c7b5872/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:00:02.147089 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b711d2e-c3ee-4a03-afff-7c88e973350f admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:02.177221 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6ece80-a3b9-4677-a98c-ad332b3dcd38 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:02.329106 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fe69a0-3959-47bd-bfc8-0159f6a09f19 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b1b651c73664d08accc6edf2f79bace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:02.329477 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b711d2e-c3ee-4a03-afff-7c88e973350f admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:02.329752 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fe69a0-3959-47bd-bfc8-0159f6a09f19 service cinder] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:02.330031 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fe69a0-3959-47bd-bfc8-0159f6a09f19 service cinder] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:02.330647 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fe69a0-3959-47bd-bfc8-0159f6a09f19 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:02.342289 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c60261-cbe6-4c75-9d1e-32c70cad2f51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:02.360858 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c60261-cbe6-4c75-9d1e-32c70cad2f51 admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:02.363945 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c60261-cbe6-4c75-9d1e-32c70cad2f51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:02.364530 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c60261-cbe6-4c75-9d1e-32c70cad2f51 admin admin] PATH_INFO: `/v3/roles` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:02.365866 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c60261-cbe6-4c75-9d1e-32c70cad2f51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:02.376062 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c60261-cbe6-4c75-9d1e-32c70cad2f51 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:02.430684 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 700/1398] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:00:02 2022] GET /identity/v3/roles => generated 1420 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 11 12:00:02.434018 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c60261-cbe6-4c75-9d1e-32c70cad2f51 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:02.489483 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c60261-cbe6-4c75-9d1e-32c70cad2f51 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:02.542073 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c1ba40-0f81-406c-9449-060a95c18234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:02.547842 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c1ba40-0f81-406c-9449-060a95c18234 admin admin] REQUEST_METHOD: `PUT` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:02.557351 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c1ba40-0f81-406c-9449-060a95c18234 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:02.558148 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c1ba40-0f81-406c-9449-060a95c18234 admin admin] PATH_INFO: `/v3/projects/54ad85255745456a98daf8c94b8a1c25/users/d5ade4db7fe943b2b7d6b33b7c7b5872/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:02.564753 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c1ba40-0f81-406c-9449-060a95c18234 admin admin] RBAC: Authorizing `identity:create_grant(project_id=54ad85255745456a98daf8c94b8a1c25, user_id=d5ade4db7fe943b2b7d6b33b7c7b5872, role_id=5cf1f4bd2fef46ab9c6eff56d52f9f69)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:02.570368 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c1ba40-0f81-406c-9449-060a95c18234 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:02.643681 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 701/1399] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 12:00:02 2022] PUT /identity/v3/projects/54ad85255745456a98daf8c94b8a1c25/users/d5ade4db7fe943b2b7d6b33b7c7b5872/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:00:02.645032 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded2bed4-ae02-48ef-b34b-18fef4636824 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:02.645613 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded2bed4-ae02-48ef-b34b-18fef4636824 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:02.646245 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded2bed4-ae02-48ef-b34b-18fef4636824 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:02.689566 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fe69a0-3959-47bd-bfc8-0159f6a09f19 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:02.691937 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fe69a0-3959-47bd-bfc8-0159f6a09f19 service cinder] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:02.733649 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 699/1400] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:00:01 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 785 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:00:02.778887 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-ded2bed4-ae02-48ef-b34b-18fef4636824 None None] MFA Rules not processed for user `d5ade4db7fe943b2b7d6b33b7c7b5872`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:00:02.781096 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-ded2bed4-ae02-48ef-b34b-18fef4636824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:03.013897 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022cdaa8-9747-4706-8870-f81289d9a33f None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:03.014416 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022cdaa8-9747-4706-8870-f81289d9a33f None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:03.015658 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022cdaa8-9747-4706-8870-f81289d9a33f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:03.116065 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-022cdaa8-9747-4706-8870-f81289d9a33f None None] MFA Rules not processed for user `41d3680dda7c4cc8aeb6d086efca0775`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:00:03.118323 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-022cdaa8-9747-4706-8870-f81289d9a33f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:03.176420 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:00:03.283840 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 702/1401] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:00:02 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 641 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 12:00:03.296516 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded2bed4-ae02-48ef-b34b-18fef4636824 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:03.578984 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded2bed4-ae02-48ef-b34b-18fef4636824 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:03.584851 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d049a9-9580-4fab-8aa9-f91be6318ab6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b1b651c73664d08accc6edf2f79bace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:03.585866 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d049a9-9580-4fab-8aa9-f91be6318ab6 service cinder] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:03.586312 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d049a9-9580-4fab-8aa9-f91be6318ab6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:03.586782 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d049a9-9580-4fab-8aa9-f91be6318ab6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:03.599017 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 700/1402] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:00:03 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 587 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:00:03.609438 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:00:03.618854 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022cdaa8-9747-4706-8870-f81289d9a33f None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:03.620467 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-022cdaa8-9747-4706-8870-f81289d9a33f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:03.622510 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d049a9-9580-4fab-8aa9-f91be6318ab6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:03.627171 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d049a9-9580-4fab-8aa9-f91be6318ab6 service cinder] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:03.645673 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 703/1403] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:00:03 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:00:03.851986 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022cdaa8-9747-4706-8870-f81289d9a33f None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:03.866838 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae49871d-8cc4-48e5-8399-3db50195fac0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bad5e1cb306b46818096940acc3810b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:03.867675 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae49871d-8cc4-48e5-8399-3db50195fac0 service nova] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:03.868026 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae49871d-8cc4-48e5-8399-3db50195fac0 service nova] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:03.868512 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae49871d-8cc4-48e5-8399-3db50195fac0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:03.887403 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae49871d-8cc4-48e5-8399-3db50195fac0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:03.892060 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae49871d-8cc4-48e5-8399-3db50195fac0 service nova] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:03.928527 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 701/1404] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:00:03 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:00:04.169034 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d049a9-9580-4fab-8aa9-f91be6318ab6 service cinder] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:04.176408 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae49871d-8cc4-48e5-8399-3db50195fac0 service nova] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:04.206705 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae49871d-8cc4-48e5-8399-3db50195fac0 service nova] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:04.225155 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d007ecb-a74d-4917-a272-ca5c24381afd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b1b651c73664d08accc6edf2f79bace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:04.232898 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d007ecb-a74d-4917-a272-ca5c24381afd service cinder] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:04.235213 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d007ecb-a74d-4917-a272-ca5c24381afd service cinder] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:04.238850 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d007ecb-a74d-4917-a272-ca5c24381afd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:04.277366 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-3d007ecb-a74d-4917-a272-ca5c24381afd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:04.299002 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d049a9-9580-4fab-8aa9-f91be6318ab6 service cinder] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:04.330262 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269e7646-39b3-4216-ab54-217c45fb8aad tempest-VolumeQuotaClassesTest-1028793100 tempest-VolumeQuotaClassesTest-1028793100-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37f8c72e7ac44e4d97f0e85646539db4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cda9f60831234c0ba6c5d67dc8cfb7b7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:04.331306 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269e7646-39b3-4216-ab54-217c45fb8aad tempest-VolumeQuotaClassesTest-1028793100 tempest-VolumeQuotaClassesTest-1028793100-project-admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:04.332213 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269e7646-39b3-4216-ab54-217c45fb8aad tempest-VolumeQuotaClassesTest-1028793100 tempest-VolumeQuotaClassesTest-1028793100-project-admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:04.332515 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269e7646-39b3-4216-ab54-217c45fb8aad tempest-VolumeQuotaClassesTest-1028793100 tempest-VolumeQuotaClassesTest-1028793100-project-admin] PATH_INFO: `/v3/domains` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:04.334712 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269e7646-39b3-4216-ab54-217c45fb8aad tempest-VolumeQuotaClassesTest-1028793100 tempest-VolumeQuotaClassesTest-1028793100-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 12:00:04.335143 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269e7646-39b3-4216-ab54-217c45fb8aad tempest-VolumeQuotaClassesTest-1028793100 tempest-VolumeQuotaClassesTest-1028793100-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:04.340172 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269e7646-39b3-4216-ab54-217c45fb8aad tempest-VolumeQuotaClassesTest-1028793100 tempest-VolumeQuotaClassesTest-1028793100-project-admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:04.371981 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 704/1405] 10.222.0.59 () {58 vars in 1153 bytes} [Tue Oct 11 12:00:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 12:00:04.373591 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269e7646-39b3-4216-ab54-217c45fb8aad tempest-VolumeQuotaClassesTest-1028793100 tempest-VolumeQuotaClassesTest-1028793100-project-admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:04.401039 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269e7646-39b3-4216-ab54-217c45fb8aad tempest-VolumeQuotaClassesTest-1028793100 tempest-VolumeQuotaClassesTest-1028793100-project-admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:04.413134 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003750d2-c53b-422e-a14a-d3822223aed8 tempest-VolumeQuotaClassesTest-1028793100 tempest-VolumeQuotaClassesTest-1028793100-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37f8c72e7ac44e4d97f0e85646539db4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cda9f60831234c0ba6c5d67dc8cfb7b7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:04.413889 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003750d2-c53b-422e-a14a-d3822223aed8 tempest-VolumeQuotaClassesTest-1028793100 tempest-VolumeQuotaClassesTest-1028793100-project-admin] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:04.414176 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003750d2-c53b-422e-a14a-d3822223aed8 tempest-VolumeQuotaClassesTest-1028793100 tempest-VolumeQuotaClassesTest-1028793100-project-admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:04.414446 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003750d2-c53b-422e-a14a-d3822223aed8 tempest-VolumeQuotaClassesTest-1028793100 tempest-VolumeQuotaClassesTest-1028793100-project-admin] PATH_INFO: `/v3/projects` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:04.417662 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003750d2-c53b-422e-a14a-d3822223aed8 tempest-VolumeQuotaClassesTest-1028793100 tempest-VolumeQuotaClassesTest-1028793100-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:04.421264 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003750d2-c53b-422e-a14a-d3822223aed8 tempest-VolumeQuotaClassesTest-1028793100 tempest-VolumeQuotaClassesTest-1028793100-project-admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:04.507309 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 705/1406] 10.222.0.59 () {60 vars in 1153 bytes} [Tue Oct 11 12:00:04 2022] POST /identity/v3/projects => generated 354 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:00:04.508407 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003750d2-c53b-422e-a14a-d3822223aed8 tempest-VolumeQuotaClassesTest-1028793100 tempest-VolumeQuotaClassesTest-1028793100-project-admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:04.587642 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d007ecb-a74d-4917-a272-ca5c24381afd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:04.590988 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d007ecb-a74d-4917-a272-ca5c24381afd service cinder] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:04.628352 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003750d2-c53b-422e-a14a-d3822223aed8 tempest-VolumeQuotaClassesTest-1028793100 tempest-VolumeQuotaClassesTest-1028793100-project-admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:04.634194 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10f4d73-9797-4465-aad3-5e862af7b0ef tempest-VolumeQuotaClassesTest-1028793100 tempest-VolumeQuotaClassesTest-1028793100-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37f8c72e7ac44e4d97f0e85646539db4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cda9f60831234c0ba6c5d67dc8cfb7b7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:04.634881 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10f4d73-9797-4465-aad3-5e862af7b0ef tempest-VolumeQuotaClassesTest-1028793100 tempest-VolumeQuotaClassesTest-1028793100-project-admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:04.635164 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10f4d73-9797-4465-aad3-5e862af7b0ef tempest-VolumeQuotaClassesTest-1028793100 tempest-VolumeQuotaClassesTest-1028793100-project-admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:04.635448 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10f4d73-9797-4465-aad3-5e862af7b0ef tempest-VolumeQuotaClassesTest-1028793100 tempest-VolumeQuotaClassesTest-1028793100-project-admin] PATH_INFO: `/v3/domains` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:04.637616 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10f4d73-9797-4465-aad3-5e862af7b0ef tempest-VolumeQuotaClassesTest-1028793100 tempest-VolumeQuotaClassesTest-1028793100-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 12:00:04.638020 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10f4d73-9797-4465-aad3-5e862af7b0ef tempest-VolumeQuotaClassesTest-1028793100 tempest-VolumeQuotaClassesTest-1028793100-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:04.640327 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10f4d73-9797-4465-aad3-5e862af7b0ef tempest-VolumeQuotaClassesTest-1028793100 tempest-VolumeQuotaClassesTest-1028793100-project-admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:04.642139 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 702/1407] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:00:04 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 467 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 11 12:00:04.665531 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 706/1408] 10.222.0.59 () {58 vars in 1153 bytes} [Tue Oct 11 12:00:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 12:00:04.776562 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d007ecb-a74d-4917-a272-ca5c24381afd service cinder] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:04.844352 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d007ecb-a74d-4917-a272-ca5c24381afd service cinder] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:04.885686 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13e21ff-8cbd-4d52-8255-0dd1163fe68e tempest-VolumeQuotaClassesTest-1028793100 tempest-VolumeQuotaClassesTest-1028793100-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37f8c72e7ac44e4d97f0e85646539db4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cda9f60831234c0ba6c5d67dc8cfb7b7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:04.886389 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13e21ff-8cbd-4d52-8255-0dd1163fe68e tempest-VolumeQuotaClassesTest-1028793100 tempest-VolumeQuotaClassesTest-1028793100-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:04.886770 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13e21ff-8cbd-4d52-8255-0dd1163fe68e tempest-VolumeQuotaClassesTest-1028793100 tempest-VolumeQuotaClassesTest-1028793100-project-admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:04.887058 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13e21ff-8cbd-4d52-8255-0dd1163fe68e tempest-VolumeQuotaClassesTest-1028793100 tempest-VolumeQuotaClassesTest-1028793100-project-admin] PATH_INFO: `/v3/projects/2c0d04553a1f4936a73fc54f85fbd198` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:04.889271 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13e21ff-8cbd-4d52-8255-0dd1163fe68e tempest-VolumeQuotaClassesTest-1028793100 tempest-VolumeQuotaClassesTest-1028793100-project-admin] RBAC: Authorizing `identity:delete_project(project_id=2c0d04553a1f4936a73fc54f85fbd198)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:04.891387 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13e21ff-8cbd-4d52-8255-0dd1163fe68e tempest-VolumeQuotaClassesTest-1028793100 tempest-VolumeQuotaClassesTest-1028793100-project-admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:04.948534 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:00:05.026206 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-e13e21ff-8cbd-4d52-8255-0dd1163fe68e tempest-VolumeQuotaClassesTest-1028793100 tempest-VolumeQuotaClassesTest-1028793100-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c0d04553a1f4936a73fc54f85fbd198', 'request_id': 'req-e13e21ff-8cbd-4d52-8255-0dd1163fe68e'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:05.048099 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-e13e21ff-8cbd-4d52-8255-0dd1163fe68e tempest-VolumeQuotaClassesTest-1028793100 tempest-VolumeQuotaClassesTest-1028793100-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c0d04553a1f4936a73fc54f85fbd198', 'request_id': 'req-e13e21ff-8cbd-4d52-8255-0dd1163fe68e'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:05.065940 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a69b8bc-d9d0-4e22-870a-91737475f6eb None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:05.066250 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a69b8bc-d9d0-4e22-870a-91737475f6eb None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:05.066814 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a69b8bc-d9d0-4e22-870a-91737475f6eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:05.077666 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-e13e21ff-8cbd-4d52-8255-0dd1163fe68e tempest-VolumeQuotaClassesTest-1028793100 tempest-VolumeQuotaClassesTest-1028793100-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c0d04553a1f4936a73fc54f85fbd198', 'request_id': 'req-e13e21ff-8cbd-4d52-8255-0dd1163fe68e'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:05.111153 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-e13e21ff-8cbd-4d52-8255-0dd1163fe68e tempest-VolumeQuotaClassesTest-1028793100 tempest-VolumeQuotaClassesTest-1028793100-project-admin] The token cache is being invalidate because project 2c0d04553a1f4936a73fc54f85fbd198 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81432) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 12:00:05.111723 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-e13e21ff-8cbd-4d52-8255-0dd1163fe68e tempest-VolumeQuotaClassesTest-1028793100 tempest-VolumeQuotaClassesTest-1028793100-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:05.115191 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-e13e21ff-8cbd-4d52-8255-0dd1163fe68e tempest-VolumeQuotaClassesTest-1028793100 tempest-VolumeQuotaClassesTest-1028793100-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:05.120539 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 703/1409] 10.222.0.59 () {58 vars in 1233 bytes} [Tue Oct 11 12:00:04 2022] DELETE /identity/v3/projects/2c0d04553a1f4936a73fc54f85fbd198 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:00:05.238716 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-9a69b8bc-d9d0-4e22-870a-91737475f6eb None None] MFA Rules not processed for user `d1619ed83ae546a7ac2219c055ca4076`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:00:05.254723 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-9a69b8bc-d9d0-4e22-870a-91737475f6eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:05.491394 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:00:05.592855 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 707/1410] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:00:05 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 528 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 12:00:05.603515 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce89c706-992c-44d2-9c84-16e6c98fd194 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:05.605661 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce89c706-992c-44d2-9c84-16e6c98fd194 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:05.606335 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce89c706-992c-44d2-9c84-16e6c98fd194 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:05.647373 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-ce89c706-992c-44d2-9c84-16e6c98fd194 None None] MFA Rules not processed for user `d1619ed83ae546a7ac2219c055ca4076`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:00:05.648743 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-ce89c706-992c-44d2-9c84-16e6c98fd194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:05.681105 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 704/1411] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:00:05 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:00:05.699189 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a69b8bc-d9d0-4e22-870a-91737475f6eb None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:05.701669 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-9a69b8bc-d9d0-4e22-870a-91737475f6eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:06.003109 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce89c706-992c-44d2-9c84-16e6c98fd194 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:06.007246 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-ce89c706-992c-44d2-9c84-16e6c98fd194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:06.083717 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a69b8bc-d9d0-4e22-870a-91737475f6eb None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:06.096826 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e7b2c2-d81d-4591-8a11-d605b89f7e6c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bad5e1cb306b46818096940acc3810b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:06.099300 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e7b2c2-d81d-4591-8a11-d605b89f7e6c service nova] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:06.099300 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e7b2c2-d81d-4591-8a11-d605b89f7e6c service nova] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:06.099300 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e7b2c2-d81d-4591-8a11-d605b89f7e6c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:06.149874 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e7b2c2-d81d-4591-8a11-d605b89f7e6c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:06.159498 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e7b2c2-d81d-4591-8a11-d605b89f7e6c service nova] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:06.222747 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 708/1412] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:00:05 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 526 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:00:06.226739 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e7b2c2-d81d-4591-8a11-d605b89f7e6c service nova] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:06.236602 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-c9e7b2c2-d81d-4591-8a11-d605b89f7e6c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:06.364530 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce89c706-992c-44d2-9c84-16e6c98fd194 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:06.388141 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3262c10a-793b-42b8-a762-fd434cc07e6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:06.393194 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3262c10a-793b-42b8-a762-fd434cc07e6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:06.393566 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3262c10a-793b-42b8-a762-fd434cc07e6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:06.393847 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3262c10a-793b-42b8-a762-fd434cc07e6b admin admin] PATH_INFO: `/v3/users/4fe1a7003a7f402c9f77cc75f161d4d5` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:06.398196 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3262c10a-793b-42b8-a762-fd434cc07e6b admin admin] RBAC: Authorizing `identity:delete_user(user_id=4fe1a7003a7f402c9f77cc75f161d4d5)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:06.409042 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3262c10a-793b-42b8-a762-fd434cc07e6b admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:06.590926 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-3262c10a-793b-42b8-a762-fd434cc07e6b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4fe1a7003a7f402c9f77cc75f161d4d5'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:06.626880 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-3262c10a-793b-42b8-a762-fd434cc07e6b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4fe1a7003a7f402c9f77cc75f161d4d5'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:06.628090 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-3262c10a-793b-42b8-a762-fd434cc07e6b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4fe1a7003a7f402c9f77cc75f161d4d5'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:06.653692 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-3262c10a-793b-42b8-a762-fd434cc07e6b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4fe1a7003a7f402c9f77cc75f161d4d5'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:06.655236 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-3262c10a-793b-42b8-a762-fd434cc07e6b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4fe1a7003a7f402c9f77cc75f161d4d5'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:06.688756 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e7b2c2-d81d-4591-8a11-d605b89f7e6c service nova] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:06.701054 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f97c56-9ac5-4f21-ab55-dc5c0a6624a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:06.702247 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f97c56-9ac5-4f21-ab55-dc5c0a6624a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:06.703242 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f97c56-9ac5-4f21-ab55-dc5c0a6624a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:06.704078 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f97c56-9ac5-4f21-ab55-dc5c0a6624a6 admin admin] PATH_INFO: `/v3/users/443b625c635944189820425cea6c5722` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:06.705853 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f97c56-9ac5-4f21-ab55-dc5c0a6624a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=443b625c635944189820425cea6c5722)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:06.715619 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 705/1413] 10.222.0.59 () {58 vars in 1224 bytes} [Tue Oct 11 12:00:06 2022] DELETE /identity/v3/users/4fe1a7003a7f402c9f77cc75f161d4d5 => generated 0 bytes in 714 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:00:06.717418 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c917c9b-bb28-4899-aa24-7afce3914c03 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:06.717963 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c917c9b-bb28-4899-aa24-7afce3914c03 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:06.718476 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c917c9b-bb28-4899-aa24-7afce3914c03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:06.721827 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f97c56-9ac5-4f21-ab55-dc5c0a6624a6 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:06.793136 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-6c917c9b-bb28-4899-aa24-7afce3914c03 None None] MFA Rules not processed for user `2fb546bc272b405699262f07ef584c7f`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:00:06.794712 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-6c917c9b-bb28-4899-aa24-7afce3914c03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:06.820024 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:00:06.878807 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-f8f97c56-9ac5-4f21-ab55-dc5c0a6624a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '443b625c635944189820425cea6c5722'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:06.929834 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-f8f97c56-9ac5-4f21-ab55-dc5c0a6624a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '443b625c635944189820425cea6c5722'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:06.946629 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-f8f97c56-9ac5-4f21-ab55-dc5c0a6624a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '443b625c635944189820425cea6c5722'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:06.947917 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-f8f97c56-9ac5-4f21-ab55-dc5c0a6624a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '443b625c635944189820425cea6c5722'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:06.978017 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:00:06.986315 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-f8f97c56-9ac5-4f21-ab55-dc5c0a6624a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '443b625c635944189820425cea6c5722'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:06.990926 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 706/1414] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:00:06 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:00:06.992719 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88a22da-806a-43ee-8fc8-0dc5bb7fafe5 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:06.997950 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88a22da-806a-43ee-8fc8-0dc5bb7fafe5 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:06.999242 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88a22da-806a-43ee-8fc8-0dc5bb7fafe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:07.008595 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 709/1415] 10.222.0.59 () {58 vars in 1224 bytes} [Tue Oct 11 12:00:06 2022] DELETE /identity/v3/users/443b625c635944189820425cea6c5722 => generated 0 bytes in 782 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:00:07.014179 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f97c56-9ac5-4f21-ab55-dc5c0a6624a6 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:07.016968 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-f8f97c56-9ac5-4f21-ab55-dc5c0a6624a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:07.076286 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-f88a22da-806a-43ee-8fc8-0dc5bb7fafe5 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:00:07.099076 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-f88a22da-806a-43ee-8fc8-0dc5bb7fafe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:07.214770 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f97c56-9ac5-4f21-ab55-dc5c0a6624a6 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:07.232787 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdca19a-fb1b-4072-adb2-3394a0c61b71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:07.233522 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdca19a-fb1b-4072-adb2-3394a0c61b71 admin admin] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:07.233923 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdca19a-fb1b-4072-adb2-3394a0c61b71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:07.235873 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdca19a-fb1b-4072-adb2-3394a0c61b71 admin admin] PATH_INFO: `/v3/projects` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:07.236665 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdca19a-fb1b-4072-adb2-3394a0c61b71 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:07.239746 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdca19a-fb1b-4072-adb2-3394a0c61b71 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:07.309828 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 707/1416] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:00:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:00:07.311620 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 710/1417] 10.222.0.59 () {60 vars in 1153 bytes} [Tue Oct 11 12:00:07 2022] POST /identity/v3/projects => generated 403 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:00:07.311951 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d79ad3-c114-4087-ac37-745647a506ac None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:07.312571 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d79ad3-c114-4087-ac37-745647a506ac None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:07.313101 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d79ad3-c114-4087-ac37-745647a506ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:07.314127 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdca19a-fb1b-4072-adb2-3394a0c61b71 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:07.328491 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-bbdca19a-fb1b-4072-adb2-3394a0c61b71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:07.341445 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-16d79ad3-c114-4087-ac37-745647a506ac None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:00:07.342667 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-16d79ad3-c114-4087-ac37-745647a506ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:07.445124 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 708/1418] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:00:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:00:07.446383 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d79ad3-c114-4087-ac37-745647a506ac None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:07.466708 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d79ad3-c114-4087-ac37-745647a506ac None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:07.475427 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728c64b1-4600-481e-b3fa-5d5d5d2da9f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:07.476573 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728c64b1-4600-481e-b3fa-5d5d5d2da9f5 admin admin] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:07.477074 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728c64b1-4600-481e-b3fa-5d5d5d2da9f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:07.477629 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728c64b1-4600-481e-b3fa-5d5d5d2da9f5 admin admin] PATH_INFO: `/v3/users` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:07.478399 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728c64b1-4600-481e-b3fa-5d5d5d2da9f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:07.481269 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728c64b1-4600-481e-b3fa-5d5d5d2da9f5 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:07.538823 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 709/1419] 10.222.0.59 () {60 vars in 1144 bytes} [Tue Oct 11 12:00:07 2022] POST /identity/v3/users => generated 308 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:00:07.542003 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728c64b1-4600-481e-b3fa-5d5d5d2da9f5 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:07.556378 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-728c64b1-4600-481e-b3fa-5d5d5d2da9f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:07.608991 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdca19a-fb1b-4072-adb2-3394a0c61b71 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:07.608991 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fa39fe-6dbb-4b61-b297-284487b1a7c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b1b651c73664d08accc6edf2f79bace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:07.608991 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fa39fe-6dbb-4b61-b297-284487b1a7c3 service cinder] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:07.608991 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fa39fe-6dbb-4b61-b297-284487b1a7c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:07.633901 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fa39fe-6dbb-4b61-b297-284487b1a7c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:07.633901 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-05fa39fe-6dbb-4b61-b297-284487b1a7c3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:07.706709 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:00:07.793772 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728c64b1-4600-481e-b3fa-5d5d5d2da9f5 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:07.812438 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcf6260-ff91-40f6-8393-1412a1dfbc87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e563fe174db4cc19ae6bf8162cc3d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:07.813407 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcf6260-ff91-40f6-8393-1412a1dfbc87 service neutron] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:07.813814 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcf6260-ff91-40f6-8393-1412a1dfbc87 service neutron] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:07.814244 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcf6260-ff91-40f6-8393-1412a1dfbc87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:07.842064 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fa39fe-6dbb-4b61-b297-284487b1a7c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:07.844307 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fa39fe-6dbb-4b61-b297-284487b1a7c3 service cinder] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:07.849922 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcf6260-ff91-40f6-8393-1412a1dfbc87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:07.852836 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcf6260-ff91-40f6-8393-1412a1dfbc87 service neutron] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:07.891664 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 710/1420] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:00:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:00:07.899126 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e45367-357f-45a3-8abd-68d2b0aa2362 None None] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:07.899710 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e45367-357f-45a3-8abd-68d2b0aa2362 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:07.900202 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e45367-357f-45a3-8abd-68d2b0aa2362 None None] PATH_INFO: `/` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:07.902667 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 711/1421] 10.222.0.59 () {54 vars in 912 bytes} [Tue Oct 11 12:00:07 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 12:00:07.904305 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e45367-357f-45a3-8abd-68d2b0aa2362 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:07.936955 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 711/1422] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:00:07 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 622 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:00:07.943631 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fa39fe-6dbb-4b61-b297-284487b1a7c3 service cinder] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:07.949746 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e45367-357f-45a3-8abd-68d2b0aa2362 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:07.963786 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b3bf47-a53c-4535-897b-d30e3e5e1195 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e563fe174db4cc19ae6bf8162cc3d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:07.964736 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b3bf47-a53c-4535-897b-d30e3e5e1195 service neutron] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:07.965146 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b3bf47-a53c-4535-897b-d30e3e5e1195 service neutron] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:07.965547 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b3bf47-a53c-4535-897b-d30e3e5e1195 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:08.002148 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fa39fe-6dbb-4b61-b297-284487b1a7c3 service cinder] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:08.007106 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b3bf47-a53c-4535-897b-d30e3e5e1195 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:08.009985 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b3bf47-a53c-4535-897b-d30e3e5e1195 service neutron] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:08.028593 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931d63d4-4ff3-4f80-80db-d15310b90f28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:08.028850 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931d63d4-4ff3-4f80-80db-d15310b90f28 admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:08.029192 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931d63d4-4ff3-4f80-80db-d15310b90f28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:08.029645 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931d63d4-4ff3-4f80-80db-d15310b90f28 admin admin] PATH_INFO: `/v3/roles` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:08.031343 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931d63d4-4ff3-4f80-80db-d15310b90f28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:08.033779 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931d63d4-4ff3-4f80-80db-d15310b90f28 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:08.050267 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 712/1423] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:00:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:00:08.073267 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e9afc8-738e-437f-93bb-4f81e1a1bea0 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:08.073665 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e9afc8-738e-437f-93bb-4f81e1a1bea0 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:08.074154 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e9afc8-738e-437f-93bb-4f81e1a1bea0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:08.086943 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 712/1424] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:00:07 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:00:08.091114 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931d63d4-4ff3-4f80-80db-d15310b90f28 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:08.140385 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931d63d4-4ff3-4f80-80db-d15310b90f28 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:08.156823 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae9b778-db59-45e5-a2ad-7bfa5e757d5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:08.160968 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae9b778-db59-45e5-a2ad-7bfa5e757d5b admin admin] REQUEST_METHOD: `PUT` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:08.161519 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae9b778-db59-45e5-a2ad-7bfa5e757d5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:08.163010 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae9b778-db59-45e5-a2ad-7bfa5e757d5b admin admin] PATH_INFO: `/v3/projects/2fa89b22c474403686edde15d32c553e/users/ebb0a9305a9246939961e643642f52b5/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:08.234480 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-f1e9afc8-738e-437f-93bb-4f81e1a1bea0 None None] MFA Rules not processed for user `2e563fe174db4cc19ae6bf8162cc3d23`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:00:08.236675 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-f1e9afc8-738e-437f-93bb-4f81e1a1bea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:08.262085 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae9b778-db59-45e5-a2ad-7bfa5e757d5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=2fa89b22c474403686edde15d32c553e, user_id=ebb0a9305a9246939961e643642f52b5, role_id=5cf1f4bd2fef46ab9c6eff56d52f9f69)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:08.274191 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 713/1425] 10.222.0.59 () {58 vars in 1009 bytes} [Tue Oct 11 12:00:08 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:00:08.277928 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae9b778-db59-45e5-a2ad-7bfa5e757d5b admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:08.348605 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:00:08.369100 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 713/1426] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 12:00:08 2022] PUT /identity/v3/projects/2fa89b22c474403686edde15d32c553e/users/ebb0a9305a9246939961e643642f52b5/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:00:08.375012 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e9afc8-738e-437f-93bb-4f81e1a1bea0 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:08.390529 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae9b778-db59-45e5-a2ad-7bfa5e757d5b admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:08.424493 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e9afc8-738e-437f-93bb-4f81e1a1bea0 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:08.635273 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae9b778-db59-45e5-a2ad-7bfa5e757d5b admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:08.642837 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e453aee0-f209-4e83-adfe-6d7fd88371f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e563fe174db4cc19ae6bf8162cc3d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:08.644295 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e453aee0-f209-4e83-adfe-6d7fd88371f5 service neutron] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:08.644900 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e453aee0-f209-4e83-adfe-6d7fd88371f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:08.645463 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e453aee0-f209-4e83-adfe-6d7fd88371f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:08.664472 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e453aee0-f209-4e83-adfe-6d7fd88371f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:08.670139 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e453aee0-f209-4e83-adfe-6d7fd88371f5 service neutron] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:08.733295 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 714/1427] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:00:08 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:00:08.773186 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7807a1d9-0fea-4b77-a05d-243c45032724 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:08.774363 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7807a1d9-0fea-4b77-a05d-243c45032724 admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:08.774900 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7807a1d9-0fea-4b77-a05d-243c45032724 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:08.775440 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7807a1d9-0fea-4b77-a05d-243c45032724 admin admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:08.776742 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7807a1d9-0fea-4b77-a05d-243c45032724 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:08.781111 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7807a1d9-0fea-4b77-a05d-243c45032724 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:08.813959 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e453aee0-f209-4e83-adfe-6d7fd88371f5 service neutron] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:08.824082 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 714/1428] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:00:08 2022] GET /identity/v3/roles => generated 1420 bytes in 450 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:00:08.871348 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e453aee0-f209-4e83-adfe-6d7fd88371f5 service neutron] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:08.894749 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebbb24b-358d-477f-9ce5-456258fb65d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:08.898650 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebbb24b-358d-477f-9ce5-456258fb65d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:08.899613 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebbb24b-358d-477f-9ce5-456258fb65d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:08.907053 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebbb24b-358d-477f-9ce5-456258fb65d2 admin admin] PATH_INFO: `/v3/projects/2fa89b22c474403686edde15d32c553e/users/ebb0a9305a9246939961e643642f52b5/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:08.916215 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebbb24b-358d-477f-9ce5-456258fb65d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2fa89b22c474403686edde15d32c553e, user_id=ebb0a9305a9246939961e643642f52b5, role_id=5cf1f4bd2fef46ab9c6eff56d52f9f69)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:08.925668 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebbb24b-358d-477f-9ce5-456258fb65d2 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:08.977928 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 715/1429] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 12:00:08 2022] PUT /identity/v3/projects/2fa89b22c474403686edde15d32c553e/users/ebb0a9305a9246939961e643642f52b5/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:00:09.121210 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9c560c-6f17-46c8-ba70-c762b87c50f7 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:09.121579 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9c560c-6f17-46c8-ba70-c762b87c50f7 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:09.122654 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9c560c-6f17-46c8-ba70-c762b87c50f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:09.225058 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-eb9c560c-6f17-46c8-ba70-c762b87c50f7 None None] MFA Rules not processed for user `c58107eee1b94541a330c26d7b82a53b`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:00:09.227727 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-eb9c560c-6f17-46c8-ba70-c762b87c50f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:09.420992 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 715/1430] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:00:09 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 12:00:09.438009 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a640052-71c6-4d91-af6f-5c0fe633fde3 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:09.439489 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a640052-71c6-4d91-af6f-5c0fe633fde3 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:09.440096 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a640052-71c6-4d91-af6f-5c0fe633fde3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:09.495896 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-0a640052-71c6-4d91-af6f-5c0fe633fde3 None None] MFA Rules not processed for user `c58107eee1b94541a330c26d7b82a53b`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:00:09.498098 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-0a640052-71c6-4d91-af6f-5c0fe633fde3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:09.514710 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 716/1431] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:00:09 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:00:09.529130 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9c560c-6f17-46c8-ba70-c762b87c50f7 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:09.530116 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-eb9c560c-6f17-46c8-ba70-c762b87c50f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:09.610197 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:00:09.800265 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9c560c-6f17-46c8-ba70-c762b87c50f7 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:09.811870 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c8ff54-8a16-42d4-8b43-83f9c33fe4e7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bad5e1cb306b46818096940acc3810b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:09.813969 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c8ff54-8a16-42d4-8b43-83f9c33fe4e7 service nova] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:09.814459 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c8ff54-8a16-42d4-8b43-83f9c33fe4e7 service nova] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:09.815003 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c8ff54-8a16-42d4-8b43-83f9c33fe4e7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:09.848742 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c8ff54-8a16-42d4-8b43-83f9c33fe4e7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:09.852013 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c8ff54-8a16-42d4-8b43-83f9c33fe4e7 service nova] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:09.871977 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 716/1432] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:00:09 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:00:09.919753 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a640052-71c6-4d91-af6f-5c0fe633fde3 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:09.927941 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-0a640052-71c6-4d91-af6f-5c0fe633fde3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:10.225932 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c8ff54-8a16-42d4-8b43-83f9c33fe4e7 service nova] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:10.228784 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-15c8ff54-8a16-42d4-8b43-83f9c33fe4e7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:10.291075 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a640052-71c6-4d91-af6f-5c0fe633fde3 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:10.321002 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c730718-a4db-4bef-8600-468826bbe047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:10.322170 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c730718-a4db-4bef-8600-468826bbe047 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:10.322678 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c730718-a4db-4bef-8600-468826bbe047 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:10.323155 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c730718-a4db-4bef-8600-468826bbe047 admin admin] PATH_INFO: `/v3/projects/81ed94c4abf74746bb798c6cf80173ca` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:10.327108 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c730718-a4db-4bef-8600-468826bbe047 admin admin] RBAC: Authorizing `identity:delete_project(project_id=81ed94c4abf74746bb798c6cf80173ca)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:10.337077 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c730718-a4db-4bef-8600-468826bbe047 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:10.450114 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c8ff54-8a16-42d4-8b43-83f9c33fe4e7 service nova] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:10.459218 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccdb71d-0df7-4d9a-b853-3ec84625b2fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:10.461340 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccdb71d-0df7-4d9a-b853-3ec84625b2fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:10.462143 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccdb71d-0df7-4d9a-b853-3ec84625b2fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:10.482271 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccdb71d-0df7-4d9a-b853-3ec84625b2fb admin admin] PATH_INFO: `/v3/projects/6ec4547f0ca04016a113f272755e718a` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:10.486109 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccdb71d-0df7-4d9a-b853-3ec84625b2fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ec4547f0ca04016a113f272755e718a)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:10.487673 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccdb71d-0df7-4d9a-b853-3ec84625b2fb admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:10.603813 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-1c730718-a4db-4bef-8600-468826bbe047 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81ed94c4abf74746bb798c6cf80173ca', 'request_id': 'req-1c730718-a4db-4bef-8600-468826bbe047'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:10.623509 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-1c730718-a4db-4bef-8600-468826bbe047 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81ed94c4abf74746bb798c6cf80173ca', 'request_id': 'req-1c730718-a4db-4bef-8600-468826bbe047'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:10.633774 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-2ccdb71d-0df7-4d9a-b853-3ec84625b2fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ec4547f0ca04016a113f272755e718a', 'request_id': 'req-2ccdb71d-0df7-4d9a-b853-3ec84625b2fb'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:10.652646 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-2ccdb71d-0df7-4d9a-b853-3ec84625b2fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ec4547f0ca04016a113f272755e718a', 'request_id': 'req-2ccdb71d-0df7-4d9a-b853-3ec84625b2fb'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:10.656410 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-1c730718-a4db-4bef-8600-468826bbe047 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81ed94c4abf74746bb798c6cf80173ca', 'request_id': 'req-1c730718-a4db-4bef-8600-468826bbe047'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:10.674619 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-2ccdb71d-0df7-4d9a-b853-3ec84625b2fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ec4547f0ca04016a113f272755e718a', 'request_id': 'req-2ccdb71d-0df7-4d9a-b853-3ec84625b2fb'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:10.699414 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-1c730718-a4db-4bef-8600-468826bbe047 admin admin] The token cache is being invalidate because project 81ed94c4abf74746bb798c6cf80173ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 12:00:10.700216 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-1c730718-a4db-4bef-8600-468826bbe047 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:10.702302 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-1c730718-a4db-4bef-8600-468826bbe047 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:10.714370 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-2ccdb71d-0df7-4d9a-b853-3ec84625b2fb admin admin] The token cache is being invalidate because project 6ec4547f0ca04016a113f272755e718a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81432) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 12:00:10.716241 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-2ccdb71d-0df7-4d9a-b853-3ec84625b2fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:10.718986 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-2ccdb71d-0df7-4d9a-b853-3ec84625b2fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:10.726281 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 717/1433] 10.222.0.59 () {58 vars in 1233 bytes} [Tue Oct 11 12:00:09 2022] DELETE /identity/v3/projects/81ed94c4abf74746bb798c6cf80173ca => generated 0 bytes in 801 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:00:10.726281 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300e77ed-ab3b-4e34-b02b-42db2e38d80a None None] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:10.726281 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300e77ed-ab3b-4e34-b02b-42db2e38d80a None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:10.726661 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300e77ed-ab3b-4e34-b02b-42db2e38d80a None None] PATH_INFO: `/` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:10.728641 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 717/1434] 10.222.0.59 () {58 vars in 1233 bytes} [Tue Oct 11 12:00:10 2022] DELETE /identity/v3/projects/6ec4547f0ca04016a113f272755e718a => generated 0 bytes in 503 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:00:10.733949 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d806c04-8600-48a8-a57d-0978850681a3 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:10.734241 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d806c04-8600-48a8-a57d-0978850681a3 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:10.734527 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d806c04-8600-48a8-a57d-0978850681a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:10.741484 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 718/1435] 10.222.0.59 () {54 vars in 912 bytes} [Tue Oct 11 12:00:10 2022] GET /identity => generated 270 bytes in 19 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 12:00:10.742682 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300e77ed-ab3b-4e34-b02b-42db2e38d80a None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:10.747308 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-300e77ed-ab3b-4e34-b02b-42db2e38d80a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:10.779833 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:00:10.795552 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-6d806c04-8600-48a8-a57d-0978850681a3 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:00:10.799079 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-6d806c04-8600-48a8-a57d-0978850681a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:10.906039 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:00:10.914631 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300e77ed-ab3b-4e34-b02b-42db2e38d80a None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:10.939105 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d51010-f233-4593-b882-88a0cde7db4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:10.939837 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d51010-f233-4593-b882-88a0cde7db4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:10.940133 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d51010-f233-4593-b882-88a0cde7db4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:10.941084 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d51010-f233-4593-b882-88a0cde7db4a admin admin] PATH_INFO: `/v3/users/37f8c72e7ac44e4d97f0e85646539db4` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:10.947326 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d51010-f233-4593-b882-88a0cde7db4a admin admin] RBAC: Authorizing `identity:delete_user(user_id=37f8c72e7ac44e4d97f0e85646539db4)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:10.955414 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d51010-f233-4593-b882-88a0cde7db4a admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:10.955619 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 718/1436] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:00:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:00:10.962900 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80501b8-3990-415d-9dd2-d9f8460ab953 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:10.963350 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80501b8-3990-415d-9dd2-d9f8460ab953 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:10.964032 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80501b8-3990-415d-9dd2-d9f8460ab953 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:11.063660 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-b80501b8-3990-415d-9dd2-d9f8460ab953 None None] MFA Rules not processed for user `2e563fe174db4cc19ae6bf8162cc3d23`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:00:11.064882 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-b80501b8-3990-415d-9dd2-d9f8460ab953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:11.087919 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-15d51010-f233-4593-b882-88a0cde7db4a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '37f8c72e7ac44e4d97f0e85646539db4'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:11.107998 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-15d51010-f233-4593-b882-88a0cde7db4a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '37f8c72e7ac44e4d97f0e85646539db4'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:11.123242 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-15d51010-f233-4593-b882-88a0cde7db4a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '37f8c72e7ac44e4d97f0e85646539db4'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:11.124083 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-15d51010-f233-4593-b882-88a0cde7db4a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '37f8c72e7ac44e4d97f0e85646539db4'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:11.151831 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-15d51010-f233-4593-b882-88a0cde7db4a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '37f8c72e7ac44e4d97f0e85646539db4'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:11.162371 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 719/1437] 10.222.0.59 () {58 vars in 1224 bytes} [Tue Oct 11 12:00:10 2022] DELETE /identity/v3/users/37f8c72e7ac44e4d97f0e85646539db4 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:00:11.164126 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee30547-d495-47ba-819f-375a733330fd None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:11.164655 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee30547-d495-47ba-819f-375a733330fd None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:11.165074 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee30547-d495-47ba-819f-375a733330fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:11.179966 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 719/1438] 10.222.0.59 () {58 vars in 1009 bytes} [Tue Oct 11 12:00:10 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:00:11.181429 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80501b8-3990-415d-9dd2-d9f8460ab953 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:11.184493 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-b80501b8-3990-415d-9dd2-d9f8460ab953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:11.203436 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-aee30547-d495-47ba-819f-375a733330fd None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:00:11.209447 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-aee30547-d495-47ba-819f-375a733330fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:11.382675 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80501b8-3990-415d-9dd2-d9f8460ab953 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:11.391699 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 720/1439] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:00:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:00:11.393103 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee30547-d495-47ba-819f-375a733330fd None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:11.394930 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-aee30547-d495-47ba-819f-375a733330fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:11.485981 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:00:11.546661 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee30547-d495-47ba-819f-375a733330fd None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:11.562905 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb816c03-29ff-427f-a23e-67e425a851c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e563fe174db4cc19ae6bf8162cc3d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:11.566106 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb816c03-29ff-427f-a23e-67e425a851c8 service neutron] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:11.569273 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb816c03-29ff-427f-a23e-67e425a851c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:11.570522 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb816c03-29ff-427f-a23e-67e425a851c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:11.593269 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb816c03-29ff-427f-a23e-67e425a851c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:11.595180 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb816c03-29ff-427f-a23e-67e425a851c8 service neutron] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:11.623240 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 721/1440] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:00:11 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:00:11.626325 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb816c03-29ff-427f-a23e-67e425a851c8 service neutron] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:11.652322 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb816c03-29ff-427f-a23e-67e425a851c8 service neutron] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:11.664799 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4949a57-936a-4935-b1ec-d0af05ba4412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:11.666288 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4949a57-936a-4935-b1ec-d0af05ba4412 admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:11.666656 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4949a57-936a-4935-b1ec-d0af05ba4412 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:11.666929 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4949a57-936a-4935-b1ec-d0af05ba4412 admin admin] PATH_INFO: `/v3/domains` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:11.669899 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4949a57-936a-4935-b1ec-d0af05ba4412 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 12:00:11.670335 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4949a57-936a-4935-b1ec-d0af05ba4412 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:11.676945 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4949a57-936a-4935-b1ec-d0af05ba4412 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:11.722099 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 722/1441] 10.222.0.59 () {58 vars in 1153 bytes} [Tue Oct 11 12:00:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 12:00:11.731087 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4949a57-936a-4935-b1ec-d0af05ba4412 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:11.733095 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e475428a-6f7e-4713-b0e1-b3d8fd33b90b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f272cdd1bcc04b4e8261ed533cfd4fd7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:11.736826 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e475428a-6f7e-4713-b0e1-b3d8fd33b90b service placement] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:11.738007 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e475428a-6f7e-4713-b0e1-b3d8fd33b90b service placement] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:11.739228 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e475428a-6f7e-4713-b0e1-b3d8fd33b90b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:11.750850 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-e475428a-6f7e-4713-b0e1-b3d8fd33b90b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:11.767545 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4949a57-936a-4935-b1ec-d0af05ba4412 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:11.785527 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17332d4-2bb6-452c-9a4b-3a0471fc0214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:11.787394 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17332d4-2bb6-452c-9a4b-3a0471fc0214 admin admin] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:11.787985 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17332d4-2bb6-452c-9a4b-3a0471fc0214 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:11.789146 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17332d4-2bb6-452c-9a4b-3a0471fc0214 admin admin] PATH_INFO: `/v3/projects` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:11.789548 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e475428a-6f7e-4713-b0e1-b3d8fd33b90b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:11.793032 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17332d4-2bb6-452c-9a4b-3a0471fc0214 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:11.796129 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e475428a-6f7e-4713-b0e1-b3d8fd33b90b service placement] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:11.798727 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17332d4-2bb6-452c-9a4b-3a0471fc0214 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:11.841273 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 720/1442] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:00:11 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 660 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:00:11.881972 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 723/1443] 10.222.0.59 () {60 vars in 1153 bytes} [Tue Oct 11 12:00:11 2022] POST /identity/v3/projects => generated 373 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:00:11.886764 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e475428a-6f7e-4713-b0e1-b3d8fd33b90b service placement] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:11.905176 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e475428a-6f7e-4713-b0e1-b3d8fd33b90b service placement] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:12.039235 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:00:12.152254 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d9de94-89b1-473c-9445-7a1a3bc7fd23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:12.153352 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d9de94-89b1-473c-9445-7a1a3bc7fd23 admin admin] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:12.153637 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d9de94-89b1-473c-9445-7a1a3bc7fd23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:12.153909 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d9de94-89b1-473c-9445-7a1a3bc7fd23 admin admin] PATH_INFO: `/v3/users` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:12.154616 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d9de94-89b1-473c-9445-7a1a3bc7fd23 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:12.156683 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d9de94-89b1-473c-9445-7a1a3bc7fd23 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:12.244401 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 721/1444] 10.222.0.59 () {60 vars in 1144 bytes} [Tue Oct 11 12:00:11 2022] POST /identity/v3/users => generated 287 bytes in 358 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 12:00:12.246051 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17332d4-2bb6-452c-9a4b-3a0471fc0214 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:12.275561 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17332d4-2bb6-452c-9a4b-3a0471fc0214 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:12.282849 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ca550b-06c4-43cb-a032-10787160d256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:12.283876 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ca550b-06c4-43cb-a032-10787160d256 admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:12.284331 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ca550b-06c4-43cb-a032-10787160d256 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:12.284763 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ca550b-06c4-43cb-a032-10787160d256 admin admin] PATH_INFO: `/v3/roles` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:12.285763 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ca550b-06c4-43cb-a032-10787160d256 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:12.287858 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ca550b-06c4-43cb-a032-10787160d256 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:12.317402 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d9de94-89b1-473c-9445-7a1a3bc7fd23 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:12.330357 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 724/1445] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:00:12 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:00:12.338390 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d9de94-89b1-473c-9445-7a1a3bc7fd23 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:12.348062 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847ea751-7a7e-4b42-b526-4d4159eb6ac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:12.349024 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847ea751-7a7e-4b42-b526-4d4159eb6ac4 admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:12.349591 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847ea751-7a7e-4b42-b526-4d4159eb6ac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:12.350122 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847ea751-7a7e-4b42-b526-4d4159eb6ac4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:12.351147 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847ea751-7a7e-4b42-b526-4d4159eb6ac4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:12.354031 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847ea751-7a7e-4b42-b526-4d4159eb6ac4 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:12.384263 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ca550b-06c4-43cb-a032-10787160d256 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:12.394790 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 722/1446] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:00:12 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:00:12.404187 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ca550b-06c4-43cb-a032-10787160d256 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:12.420725 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e759059-2ea4-4e56-a2de-bc6c9fe84678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:12.424212 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e759059-2ea4-4e56-a2de-bc6c9fe84678 admin admin] REQUEST_METHOD: `PUT` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:12.424813 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e759059-2ea4-4e56-a2de-bc6c9fe84678 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:12.424813 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e759059-2ea4-4e56-a2de-bc6c9fe84678 admin admin] PATH_INFO: `/v3/projects/85a10ff66bc5457899b9abae9b27b903/users/456d8be3a30840a19a3cdb8b82662bff/roles/de2d2bd26fff4748a79990f97a107164` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:12.465919 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e759059-2ea4-4e56-a2de-bc6c9fe84678 admin admin] RBAC: Authorizing `identity:create_grant(project_id=85a10ff66bc5457899b9abae9b27b903, user_id=456d8be3a30840a19a3cdb8b82662bff, role_id=de2d2bd26fff4748a79990f97a107164)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:12.474459 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e759059-2ea4-4e56-a2de-bc6c9fe84678 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:12.531243 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 725/1447] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 12:00:12 2022] PUT /identity/v3/projects/85a10ff66bc5457899b9abae9b27b903/users/456d8be3a30840a19a3cdb8b82662bff/roles/de2d2bd26fff4748a79990f97a107164 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:00:12.533616 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23247d43-5cd8-41ac-b9d9-927dbf86b736 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:12.534617 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23247d43-5cd8-41ac-b9d9-927dbf86b736 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:12.535347 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23247d43-5cd8-41ac-b9d9-927dbf86b736 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:12.689163 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-23247d43-5cd8-41ac-b9d9-927dbf86b736 None None] MFA Rules not processed for user `456d8be3a30840a19a3cdb8b82662bff`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:00:12.694667 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-23247d43-5cd8-41ac-b9d9-927dbf86b736 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:12.734249 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e759059-2ea4-4e56-a2de-bc6c9fe84678 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:12.736854 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-5e759059-2ea4-4e56-a2de-bc6c9fe84678 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:12.933957 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e759059-2ea4-4e56-a2de-bc6c9fe84678 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:12.955992 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfa4b56-d38a-4290-bcba-3c1c5a04a416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:12.956261 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfa4b56-d38a-4290-bcba-3c1c5a04a416 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:12.956669 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfa4b56-d38a-4290-bcba-3c1c5a04a416 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:12.957774 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfa4b56-d38a-4290-bcba-3c1c5a04a416 admin admin] PATH_INFO: `/v3/projects/cda9f60831234c0ba6c5d67dc8cfb7b7` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:12.958676 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfa4b56-d38a-4290-bcba-3c1c5a04a416 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cda9f60831234c0ba6c5d67dc8cfb7b7)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:12.970295 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfa4b56-d38a-4290-bcba-3c1c5a04a416 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:13.056370 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 723/1448] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:00:12 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 525 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:00:13.083454 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23247d43-5cd8-41ac-b9d9-927dbf86b736 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:13.086761 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-23247d43-5cd8-41ac-b9d9-927dbf86b736 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:13.185161 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-7dfa4b56-d38a-4290-bcba-3c1c5a04a416 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cda9f60831234c0ba6c5d67dc8cfb7b7', 'request_id': 'req-7dfa4b56-d38a-4290-bcba-3c1c5a04a416'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:13.205642 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-7dfa4b56-d38a-4290-bcba-3c1c5a04a416 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cda9f60831234c0ba6c5d67dc8cfb7b7', 'request_id': 'req-7dfa4b56-d38a-4290-bcba-3c1c5a04a416'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:13.237127 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-7dfa4b56-d38a-4290-bcba-3c1c5a04a416 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cda9f60831234c0ba6c5d67dc8cfb7b7', 'request_id': 'req-7dfa4b56-d38a-4290-bcba-3c1c5a04a416'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:13.239297 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:00:13.313390 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-7dfa4b56-d38a-4290-bcba-3c1c5a04a416 admin admin] The token cache is being invalidate because project cda9f60831234c0ba6c5d67dc8cfb7b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 12:00:13.314497 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-7dfa4b56-d38a-4290-bcba-3c1c5a04a416 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:13.317536 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-7dfa4b56-d38a-4290-bcba-3c1c5a04a416 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:13.335765 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 726/1449] 10.222.0.59 () {58 vars in 1233 bytes} [Tue Oct 11 12:00:12 2022] DELETE /identity/v3/projects/cda9f60831234c0ba6c5d67dc8cfb7b7 => generated 0 bytes in 593 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 12:00:13.335765 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7a7dbf-e039-48cd-8dc5-fdb379524618 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:13.335765 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7a7dbf-e039-48cd-8dc5-fdb379524618 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:13.335765 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7a7dbf-e039-48cd-8dc5-fdb379524618 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:13.425492 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-ac7a7dbf-e039-48cd-8dc5-fdb379524618 None None] MFA Rules not processed for user `c58107eee1b94541a330c26d7b82a53b`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:00:13.426539 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23247d43-5cd8-41ac-b9d9-927dbf86b736 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:13.430333 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-ac7a7dbf-e039-48cd-8dc5-fdb379524618 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:13.518860 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:00:13.603994 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 727/1450] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:00:13 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:00:13.605686 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f629f61-a6a0-4f7c-85dc-8e2af3725ddb None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:13.607442 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f629f61-a6a0-4f7c-85dc-8e2af3725ddb None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:13.607442 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f629f61-a6a0-4f7c-85dc-8e2af3725ddb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:13.628836 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fed550-1661-4cfc-a4d3-ba3f445d6b7f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bad5e1cb306b46818096940acc3810b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:13.629075 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fed550-1661-4cfc-a4d3-ba3f445d6b7f service nova] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:13.629819 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fed550-1661-4cfc-a4d3-ba3f445d6b7f service nova] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:13.630816 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fed550-1661-4cfc-a4d3-ba3f445d6b7f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:13.640218 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-79fed550-1661-4cfc-a4d3-ba3f445d6b7f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:13.660304 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-2f629f61-a6a0-4f7c-85dc-8e2af3725ddb None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:00:13.667403 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-2f629f61-a6a0-4f7c-85dc-8e2af3725ddb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:13.902319 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fed550-1661-4cfc-a4d3-ba3f445d6b7f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:13.905070 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fed550-1661-4cfc-a4d3-ba3f445d6b7f service nova] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:13.943045 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 724/1451] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:00:13 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 861 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:00:13.946191 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 728/1452] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:00:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 342 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:00:13.962910 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9116816c-1e6c-4b52-8660-9cbedc604bff None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:13.964687 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9116816c-1e6c-4b52-8660-9cbedc604bff None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:13.968748 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9116816c-1e6c-4b52-8660-9cbedc604bff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:14.225328 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6903020d-6613-4a54-b8e6-c57aa74c8c7c None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:14.225651 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6903020d-6613-4a54-b8e6-c57aa74c8c7c None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:14.226020 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6903020d-6613-4a54-b8e6-c57aa74c8c7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:14.333105 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-6903020d-6613-4a54-b8e6-c57aa74c8c7c None None] MFA Rules not processed for user `41d3680dda7c4cc8aeb6d086efca0775`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:00:14.339174 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-6903020d-6613-4a54-b8e6-c57aa74c8c7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:14.649442 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 729/1453] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:00:14 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 425 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:00:14.651318 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c8e919-2d95-4d9a-b23f-85a568cc1516 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:14.651640 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c8e919-2d95-4d9a-b23f-85a568cc1516 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:14.654989 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c8e919-2d95-4d9a-b23f-85a568cc1516 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:14.734697 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-f9c8e919-2d95-4d9a-b23f-85a568cc1516 None None] MFA Rules not processed for user `ebb0a9305a9246939961e643642f52b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:00:14.736333 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-f9c8e919-2d95-4d9a-b23f-85a568cc1516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:14.836781 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-9116816c-1e6c-4b52-8660-9cbedc604bff None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:00:14.842264 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-9116816c-1e6c-4b52-8660-9cbedc604bff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:14.854434 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:00:14.879106 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 725/1454] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:00:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 915 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:00:14.880987 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9116816c-1e6c-4b52-8660-9cbedc604bff None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:14.911295 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9116816c-1e6c-4b52-8660-9cbedc604bff None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:14.933503 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770727f5-214a-4443-910b-f9cf734e28f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:14.934862 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770727f5-214a-4443-910b-f9cf734e28f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:14.935511 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770727f5-214a-4443-910b-f9cf734e28f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:14.936160 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770727f5-214a-4443-910b-f9cf734e28f2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:14.937569 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770727f5-214a-4443-910b-f9cf734e28f2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 12:00:14.938339 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770727f5-214a-4443-910b-f9cf734e28f2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:14.947268 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770727f5-214a-4443-910b-f9cf734e28f2 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:14.950736 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 730/1455] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:00:14 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:00:15.026269 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 726/1456] 10.222.0.59 () {58 vars in 1153 bytes} [Tue Oct 11 12:00:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 12:00:15.033100 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c8e919-2d95-4d9a-b23f-85a568cc1516 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:15.058484 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c8e919-2d95-4d9a-b23f-85a568cc1516 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:15.098322 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0042289f-bdd7-46ee-85b0-6cec364731c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:15.101527 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0042289f-bdd7-46ee-85b0-6cec364731c2 admin admin] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:15.105177 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0042289f-bdd7-46ee-85b0-6cec364731c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:15.105687 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0042289f-bdd7-46ee-85b0-6cec364731c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:15.106936 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0042289f-bdd7-46ee-85b0-6cec364731c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:15.114688 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0042289f-bdd7-46ee-85b0-6cec364731c2 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:15.272964 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 731/1457] 10.222.0.59 () {60 vars in 1153 bytes} [Tue Oct 11 12:00:15 2022] POST /identity/v3/projects => generated 381 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:00:15.279634 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770727f5-214a-4443-910b-f9cf734e28f2 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:15.293292 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770727f5-214a-4443-910b-f9cf734e28f2 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:15.665469 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18b1ddd-f7d7-4e5d-a422-513bb2e789f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:15.675732 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18b1ddd-f7d7-4e5d-a422-513bb2e789f8 admin admin] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:15.680417 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18b1ddd-f7d7-4e5d-a422-513bb2e789f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:15.681235 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18b1ddd-f7d7-4e5d-a422-513bb2e789f8 admin admin] PATH_INFO: `/v3/users` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:15.682647 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18b1ddd-f7d7-4e5d-a422-513bb2e789f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:15.686674 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18b1ddd-f7d7-4e5d-a422-513bb2e789f8 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:15.867155 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0042289f-bdd7-46ee-85b0-6cec364731c2 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:15.868106 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 727/1458] 10.222.0.59 () {60 vars in 1144 bytes} [Tue Oct 11 12:00:15 2022] POST /identity/v3/users => generated 291 bytes in 589 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:00:15.887187 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0042289f-bdd7-46ee-85b0-6cec364731c2 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:15.918774 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8662fdd3-8cff-430a-b8de-a1180a69a992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:15.920963 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8662fdd3-8cff-430a-b8de-a1180a69a992 admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:15.922837 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8662fdd3-8cff-430a-b8de-a1180a69a992 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:15.923476 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8662fdd3-8cff-430a-b8de-a1180a69a992 admin admin] PATH_INFO: `/v3/roles` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:15.927162 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8662fdd3-8cff-430a-b8de-a1180a69a992 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:15.931261 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8662fdd3-8cff-430a-b8de-a1180a69a992 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:15.958187 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 732/1459] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:00:15 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:00:15.963512 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18b1ddd-f7d7-4e5d-a422-513bb2e789f8 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:15.995254 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18b1ddd-f7d7-4e5d-a422-513bb2e789f8 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:16.018671 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e9711e-5155-4129-bd20-e4b7ee9c77df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:16.019838 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e9711e-5155-4129-bd20-e4b7ee9c77df admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:16.020223 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e9711e-5155-4129-bd20-e4b7ee9c77df admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:16.020593 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e9711e-5155-4129-bd20-e4b7ee9c77df admin admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:16.021388 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e9711e-5155-4129-bd20-e4b7ee9c77df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:16.023228 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e9711e-5155-4129-bd20-e4b7ee9c77df admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:16.054190 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 728/1460] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:00:15 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:00:16.059404 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8662fdd3-8cff-430a-b8de-a1180a69a992 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:16.097512 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8662fdd3-8cff-430a-b8de-a1180a69a992 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:16.121357 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe96fec-9eda-4fc8-a409-22e6511fee38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:16.122460 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe96fec-9eda-4fc8-a409-22e6511fee38 admin admin] REQUEST_METHOD: `PUT` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:16.122939 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe96fec-9eda-4fc8-a409-22e6511fee38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:16.123401 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe96fec-9eda-4fc8-a409-22e6511fee38 admin admin] PATH_INFO: `/v3/projects/14d8e0913e38452c896f11531145f961/users/93be6832afde4bb595512e6806157101/roles/de2d2bd26fff4748a79990f97a107164` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:16.211880 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe96fec-9eda-4fc8-a409-22e6511fee38 admin admin] RBAC: Authorizing `identity:create_grant(project_id=14d8e0913e38452c896f11531145f961, user_id=93be6832afde4bb595512e6806157101, role_id=de2d2bd26fff4748a79990f97a107164)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:16.217138 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe96fec-9eda-4fc8-a409-22e6511fee38 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:16.277336 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 733/1461] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 12:00:16 2022] PUT /identity/v3/projects/14d8e0913e38452c896f11531145f961/users/93be6832afde4bb595512e6806157101/roles/de2d2bd26fff4748a79990f97a107164 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 12:00:16.296184 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9972acc4-4f37-439c-8fa0-6763dd6f6a96 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:16.297183 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9972acc4-4f37-439c-8fa0-6763dd6f6a96 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:16.297908 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9972acc4-4f37-439c-8fa0-6763dd6f6a96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:16.343362 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe96fec-9eda-4fc8-a409-22e6511fee38 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:16.346214 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-0fe96fec-9eda-4fc8-a409-22e6511fee38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:16.376904 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-9972acc4-4f37-439c-8fa0-6763dd6f6a96 None None] MFA Rules not processed for user `93be6832afde4bb595512e6806157101`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:00:16.378540 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-9972acc4-4f37-439c-8fa0-6763dd6f6a96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:16.421640 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:00:16.529293 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 729/1462] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:00:16 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:00:16.535332 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9972acc4-4f37-439c-8fa0-6763dd6f6a96 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:16.592930 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe96fec-9eda-4fc8-a409-22e6511fee38 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:16.600837 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b9b484-0582-4c5a-9287-29bab87096ca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b1b651c73664d08accc6edf2f79bace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:16.602641 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b9b484-0582-4c5a-9287-29bab87096ca service cinder] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:16.603127 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b9b484-0582-4c5a-9287-29bab87096ca service cinder] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:16.603542 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b9b484-0582-4c5a-9287-29bab87096ca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:16.607718 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-a2b9b484-0582-4c5a-9287-29bab87096ca service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:16.702141 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b9b484-0582-4c5a-9287-29bab87096ca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:16.703882 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b9b484-0582-4c5a-9287-29bab87096ca service cinder] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:16.710203 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9972acc4-4f37-439c-8fa0-6763dd6f6a96 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:16.727427 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 734/1463] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:00:16 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:00:16.733889 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c864cae-3486-48b3-8c90-c60227f3a530 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:16.735191 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c864cae-3486-48b3-8c90-c60227f3a530 admin admin] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:16.735732 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c864cae-3486-48b3-8c90-c60227f3a530 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:16.737090 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c864cae-3486-48b3-8c90-c60227f3a530 admin admin] PATH_INFO: `/v3/projects` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:16.738794 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c864cae-3486-48b3-8c90-c60227f3a530 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:16.745437 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c864cae-3486-48b3-8c90-c60227f3a530 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:16.865809 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 730/1464] 10.222.0.59 () {60 vars in 1153 bytes} [Tue Oct 11 12:00:16 2022] POST /identity/v3/projects => generated 379 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:00:16.871115 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b9b484-0582-4c5a-9287-29bab87096ca service cinder] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:16.892087 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b9b484-0582-4c5a-9287-29bab87096ca service cinder] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:16.999466 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:00:17.061611 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a9ed8f-34b3-44ef-9ddf-96bfbd2f2e66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:17.062821 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a9ed8f-34b3-44ef-9ddf-96bfbd2f2e66 admin admin] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:17.063264 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a9ed8f-34b3-44ef-9ddf-96bfbd2f2e66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:17.063889 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a9ed8f-34b3-44ef-9ddf-96bfbd2f2e66 admin admin] PATH_INFO: `/v3/users` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:17.065391 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a9ed8f-34b3-44ef-9ddf-96bfbd2f2e66 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:17.074458 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a9ed8f-34b3-44ef-9ddf-96bfbd2f2e66 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:17.144986 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 735/1465] 10.222.0.59 () {60 vars in 1144 bytes} [Tue Oct 11 12:00:16 2022] POST /identity/v3/users => generated 296 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:00:17.147041 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c864cae-3486-48b3-8c90-c60227f3a530 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:17.186730 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c864cae-3486-48b3-8c90-c60227f3a530 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:17.206845 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0b92d5-6a90-4373-a56f-f69c143adf12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:17.208326 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0b92d5-6a90-4373-a56f-f69c143adf12 admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:17.208977 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0b92d5-6a90-4373-a56f-f69c143adf12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:17.209671 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0b92d5-6a90-4373-a56f-f69c143adf12 admin admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:17.211439 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0b92d5-6a90-4373-a56f-f69c143adf12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:17.215384 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0b92d5-6a90-4373-a56f-f69c143adf12 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:17.267471 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 731/1466] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:00:17 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:00:17.271616 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a9ed8f-34b3-44ef-9ddf-96bfbd2f2e66 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:17.304838 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a9ed8f-34b3-44ef-9ddf-96bfbd2f2e66 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:17.330624 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a93a15d-1b01-41ed-a4fe-902e45b72d86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:17.332120 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a93a15d-1b01-41ed-a4fe-902e45b72d86 admin admin] REQUEST_METHOD: `PUT` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:17.332695 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a93a15d-1b01-41ed-a4fe-902e45b72d86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:17.333422 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a93a15d-1b01-41ed-a4fe-902e45b72d86 admin admin] PATH_INFO: `/v3/projects/c6a60db4ccdd4ef7a10e481abc983575/users/abd03b717caa41dc953f6874335b7551/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:17.375303 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a93a15d-1b01-41ed-a4fe-902e45b72d86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6a60db4ccdd4ef7a10e481abc983575, user_id=abd03b717caa41dc953f6874335b7551, role_id=5cf1f4bd2fef46ab9c6eff56d52f9f69)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:17.378318 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a93a15d-1b01-41ed-a4fe-902e45b72d86 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:17.413837 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 736/1467] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 12:00:17 2022] PUT /identity/v3/projects/c6a60db4ccdd4ef7a10e481abc983575/users/abd03b717caa41dc953f6874335b7551/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:00:17.415660 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0b92d5-6a90-4373-a56f-f69c143adf12 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:17.465117 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0b92d5-6a90-4373-a56f-f69c143adf12 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:17.640204 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6385853-491b-499b-95ea-ad4e94622257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:17.640854 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6385853-491b-499b-95ea-ad4e94622257 admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:17.641135 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6385853-491b-499b-95ea-ad4e94622257 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:17.641458 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6385853-491b-499b-95ea-ad4e94622257 admin admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:17.642201 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6385853-491b-499b-95ea-ad4e94622257 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:17.644014 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6385853-491b-499b-95ea-ad4e94622257 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:17.660377 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 732/1468] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:00:17 2022] GET /identity/v3/roles => generated 1420 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:00:17.662127 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a93a15d-1b01-41ed-a4fe-902e45b72d86 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:17.676107 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a93a15d-1b01-41ed-a4fe-902e45b72d86 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:17.683468 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cd690d-aa19-44f6-a6a7-9e62cbbf2dcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:17.684687 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cd690d-aa19-44f6-a6a7-9e62cbbf2dcd admin admin] REQUEST_METHOD: `PUT` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:17.685160 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cd690d-aa19-44f6-a6a7-9e62cbbf2dcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:17.686112 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cd690d-aa19-44f6-a6a7-9e62cbbf2dcd admin admin] PATH_INFO: `/v3/projects/c6a60db4ccdd4ef7a10e481abc983575/users/abd03b717caa41dc953f6874335b7551/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:17.689554 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cd690d-aa19-44f6-a6a7-9e62cbbf2dcd admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6a60db4ccdd4ef7a10e481abc983575, user_id=abd03b717caa41dc953f6874335b7551, role_id=5cf1f4bd2fef46ab9c6eff56d52f9f69)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:17.692734 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cd690d-aa19-44f6-a6a7-9e62cbbf2dcd admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:17.723920 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 737/1469] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 12:00:17 2022] PUT /identity/v3/projects/c6a60db4ccdd4ef7a10e481abc983575/users/abd03b717caa41dc953f6874335b7551/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:00:17.727240 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10796107-2a0a-4adc-8af7-fa4df53522e3 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:17.727865 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10796107-2a0a-4adc-8af7-fa4df53522e3 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:17.728472 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10796107-2a0a-4adc-8af7-fa4df53522e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:17.802684 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-10796107-2a0a-4adc-8af7-fa4df53522e3 None None] MFA Rules not processed for user `abd03b717caa41dc953f6874335b7551`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:00:17.804520 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-10796107-2a0a-4adc-8af7-fa4df53522e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:17.830477 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:00:17.919554 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 733/1470] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:00:17 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 12:00:17.932510 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cd690d-aa19-44f6-a6a7-9e62cbbf2dcd admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:17.936592 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-99cd690d-aa19-44f6-a6a7-9e62cbbf2dcd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:18.054907 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5e10ef-f0ff-49e1-808e-74eedfb01b4a None None] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:18.055535 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5e10ef-f0ff-49e1-808e-74eedfb01b4a None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:18.055949 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5e10ef-f0ff-49e1-808e-74eedfb01b4a None None] PATH_INFO: `/` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:18.059913 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 734/1471] 10.222.0.59 () {54 vars in 921 bytes} [Tue Oct 11 12:00:18 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 12:00:18.089389 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cd690d-aa19-44f6-a6a7-9e62cbbf2dcd admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:18.101181 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e09f399-b9d2-4bcf-a599-bfbde4d37682 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b1b651c73664d08accc6edf2f79bace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:18.101868 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e09f399-b9d2-4bcf-a599-bfbde4d37682 service cinder] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:18.102142 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e09f399-b9d2-4bcf-a599-bfbde4d37682 service cinder] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:18.102494 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e09f399-b9d2-4bcf-a599-bfbde4d37682 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:18.125972 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e09f399-b9d2-4bcf-a599-bfbde4d37682 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:18.127066 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e09f399-b9d2-4bcf-a599-bfbde4d37682 service cinder] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:18.151372 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f81d59-e956-4d62-91f2-9e26ed1782ef None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:18.151821 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f81d59-e956-4d62-91f2-9e26ed1782ef None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:18.151898 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f81d59-e956-4d62-91f2-9e26ed1782ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:18.155591 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 738/1472] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:00:17 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:00:18.221694 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-75f81d59-e956-4d62-91f2-9e26ed1782ef None None] MFA Rules not processed for user `f272cdd1bcc04b4e8261ed533cfd4fd7`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:00:18.226576 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-75f81d59-e956-4d62-91f2-9e26ed1782ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:18.317746 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e09f399-b9d2-4bcf-a599-bfbde4d37682 service cinder] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:18.334829 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e09f399-b9d2-4bcf-a599-bfbde4d37682 service cinder] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:18.350875 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f5131e-ab19-473c-9c34-b0c5d643a907 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b1b651c73664d08accc6edf2f79bace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:18.352327 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f5131e-ab19-473c-9c34-b0c5d643a907 service cinder] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:18.353176 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f5131e-ab19-473c-9c34-b0c5d643a907 service cinder] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:18.353356 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f5131e-ab19-473c-9c34-b0c5d643a907 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:18.513580 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 735/1473] 10.222.0.59 () {58 vars in 1018 bytes} [Tue Oct 11 12:00:18 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 364 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:00:18.555490 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f5131e-ab19-473c-9c34-b0c5d643a907 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:18.556990 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f5131e-ab19-473c-9c34-b0c5d643a907 service cinder] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:18.600247 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 739/1474] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:00:18 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:00:18.603009 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f81d59-e956-4d62-91f2-9e26ed1782ef None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:18.607172 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-75f81d59-e956-4d62-91f2-9e26ed1782ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:18.636148 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f81d59-e956-4d62-91f2-9e26ed1782ef None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:18.651322 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9ee988-6c02-4f63-a47a-3a5b24080d9f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f272cdd1bcc04b4e8261ed533cfd4fd7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:18.651925 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9ee988-6c02-4f63-a47a-3a5b24080d9f service placement] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:18.658520 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9ee988-6c02-4f63-a47a-3a5b24080d9f service placement] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:18.658895 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9ee988-6c02-4f63-a47a-3a5b24080d9f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:18.681148 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9ee988-6c02-4f63-a47a-3a5b24080d9f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:18.686858 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9ee988-6c02-4f63-a47a-3a5b24080d9f service placement] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:18.746317 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 736/1475] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:00:18 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:00:19.605054 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f5131e-ab19-473c-9c34-b0c5d643a907 service cinder] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:19.659486 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:00:19.761595 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f5131e-ab19-473c-9c34-b0c5d643a907 service cinder] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:19.784027 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0fdb20-18f2-479c-a832-4a483ac34e39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:19.785338 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0fdb20-18f2-479c-a832-4a483ac34e39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:19.786426 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0fdb20-18f2-479c-a832-4a483ac34e39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:19.787070 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0fdb20-18f2-479c-a832-4a483ac34e39 admin admin] PATH_INFO: `/v3/users/93be6832afde4bb595512e6806157101` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:19.792452 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0fdb20-18f2-479c-a832-4a483ac34e39 admin admin] RBAC: Authorizing `identity:delete_user(user_id=93be6832afde4bb595512e6806157101)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:19.796138 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0fdb20-18f2-479c-a832-4a483ac34e39 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:19.881351 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-8a0fdb20-18f2-479c-a832-4a483ac34e39 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93be6832afde4bb595512e6806157101'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:19.903844 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-8a0fdb20-18f2-479c-a832-4a483ac34e39 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93be6832afde4bb595512e6806157101'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:19.928406 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-8a0fdb20-18f2-479c-a832-4a483ac34e39 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93be6832afde4bb595512e6806157101'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:19.928406 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-8a0fdb20-18f2-479c-a832-4a483ac34e39 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93be6832afde4bb595512e6806157101'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:19.937162 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-8a0fdb20-18f2-479c-a832-4a483ac34e39 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93be6832afde4bb595512e6806157101'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:19.946629 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 740/1476] 10.222.0.59 () {58 vars in 1224 bytes} [Tue Oct 11 12:00:19 2022] DELETE /identity/v3/users/93be6832afde4bb595512e6806157101 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 12:00:19.955347 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c1dc3a-4102-4eee-bca5-8f9e26855322 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:19.969962 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c1dc3a-4102-4eee-bca5-8f9e26855322 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:19.971578 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c1dc3a-4102-4eee-bca5-8f9e26855322 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:20.020260 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0fdb20-18f2-479c-a832-4a483ac34e39 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:20.021915 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-8a0fdb20-18f2-479c-a832-4a483ac34e39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:20.034203 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-91c1dc3a-4102-4eee-bca5-8f9e26855322 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:00:20.046094 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-91c1dc3a-4102-4eee-bca5-8f9e26855322 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:20.169766 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 737/1477] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:00:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:00:20.186034 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c1dc3a-4102-4eee-bca5-8f9e26855322 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:20.187952 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-91c1dc3a-4102-4eee-bca5-8f9e26855322 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:20.192461 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0fdb20-18f2-479c-a832-4a483ac34e39 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:20.212480 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58edcf83-0a93-4af2-88ba-b9fcefdf4957 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:20.213389 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58edcf83-0a93-4af2-88ba-b9fcefdf4957 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:20.213674 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58edcf83-0a93-4af2-88ba-b9fcefdf4957 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:20.213947 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58edcf83-0a93-4af2-88ba-b9fcefdf4957 admin admin] PATH_INFO: `/v3/users/2fb546bc272b405699262f07ef584c7f` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:20.215209 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58edcf83-0a93-4af2-88ba-b9fcefdf4957 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2fb546bc272b405699262f07ef584c7f)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:20.217231 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58edcf83-0a93-4af2-88ba-b9fcefdf4957 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:20.338769 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:00:20.353767 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-58edcf83-0a93-4af2-88ba-b9fcefdf4957 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2fb546bc272b405699262f07ef584c7f'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:20.398443 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-58edcf83-0a93-4af2-88ba-b9fcefdf4957 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2fb546bc272b405699262f07ef584c7f'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:20.401073 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:00:20.421158 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-58edcf83-0a93-4af2-88ba-b9fcefdf4957 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2fb546bc272b405699262f07ef584c7f'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:20.422077 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-58edcf83-0a93-4af2-88ba-b9fcefdf4957 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2fb546bc272b405699262f07ef584c7f'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:20.444445 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c1dc3a-4102-4eee-bca5-8f9e26855322 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:20.453732 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4d70fe-a006-4c73-901d-5ef6b2987c0c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e563fe174db4cc19ae6bf8162cc3d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:20.455497 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4d70fe-a006-4c73-901d-5ef6b2987c0c service neutron] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:20.455977 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-58edcf83-0a93-4af2-88ba-b9fcefdf4957 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2fb546bc272b405699262f07ef584c7f'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:20.456341 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4d70fe-a006-4c73-901d-5ef6b2987c0c service neutron] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:20.456940 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4d70fe-a006-4c73-901d-5ef6b2987c0c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:20.461184 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-7e4d70fe-a006-4c73-901d-5ef6b2987c0c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:20.487321 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 741/1478] 10.222.0.59 () {58 vars in 1224 bytes} [Tue Oct 11 12:00:20 2022] DELETE /identity/v3/users/2fb546bc272b405699262f07ef584c7f => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 12:00:20.511980 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4d70fe-a006-4c73-901d-5ef6b2987c0c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:20.513652 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4d70fe-a006-4c73-901d-5ef6b2987c0c service neutron] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:20.542987 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 738/1479] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:00:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:00:21.148680 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58edcf83-0a93-4af2-88ba-b9fcefdf4957 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:21.157850 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-58edcf83-0a93-4af2-88ba-b9fcefdf4957 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:21.383092 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58edcf83-0a93-4af2-88ba-b9fcefdf4957 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:21.390146 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61de8446-2cab-4094-9e3d-132771e082df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:21.391152 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61de8446-2cab-4094-9e3d-132771e082df admin admin] REQUEST_METHOD: `DELETE` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:21.391576 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61de8446-2cab-4094-9e3d-132771e082df admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:21.391990 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61de8446-2cab-4094-9e3d-132771e082df admin admin] PATH_INFO: `/v3/projects/c9aa6aaab31e4dc5b5e7aa6ea8b91b0a` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:21.393846 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61de8446-2cab-4094-9e3d-132771e082df admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9aa6aaab31e4dc5b5e7aa6ea8b91b0a)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:21.396490 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61de8446-2cab-4094-9e3d-132771e082df admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:21.593200 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-61de8446-2cab-4094-9e3d-132771e082df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9aa6aaab31e4dc5b5e7aa6ea8b91b0a', 'request_id': 'req-61de8446-2cab-4094-9e3d-132771e082df'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:21.603006 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-61de8446-2cab-4094-9e3d-132771e082df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9aa6aaab31e4dc5b5e7aa6ea8b91b0a', 'request_id': 'req-61de8446-2cab-4094-9e3d-132771e082df'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:21.620820 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-61de8446-2cab-4094-9e3d-132771e082df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9aa6aaab31e4dc5b5e7aa6ea8b91b0a', 'request_id': 'req-61de8446-2cab-4094-9e3d-132771e082df'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:21.645749 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-61de8446-2cab-4094-9e3d-132771e082df admin admin] The token cache is being invalidate because project c9aa6aaab31e4dc5b5e7aa6ea8b91b0a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 12:00:21.646098 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-61de8446-2cab-4094-9e3d-132771e082df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:21.646916 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-61de8446-2cab-4094-9e3d-132771e082df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:21.656079 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4d70fe-a006-4c73-901d-5ef6b2987c0c service neutron] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:21.658823 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 742/1480] 10.222.0.59 () {58 vars in 1233 bytes} [Tue Oct 11 12:00:21 2022] DELETE /identity/v3/projects/c9aa6aaab31e4dc5b5e7aa6ea8b91b0a => generated 0 bytes in 506 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:00:21.667644 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4d70fe-a006-4c73-901d-5ef6b2987c0c service neutron] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:21.708792 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e61403e-78b7-4e97-9b20-181a6db99829 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:21.709091 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e61403e-78b7-4e97-9b20-181a6db99829 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:21.709355 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e61403e-78b7-4e97-9b20-181a6db99829 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:21.865507 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-6e61403e-78b7-4e97-9b20-181a6db99829 None None] MFA Rules not processed for user `8a19eceb62ce4157a4c5fe1866a89df9`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:00:21.868309 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-6e61403e-78b7-4e97-9b20-181a6db99829 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:21.879604 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740dbbef-8db4-4d58-995e-ac96039f4014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:21.881001 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740dbbef-8db4-4d58-995e-ac96039f4014 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:21.882134 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740dbbef-8db4-4d58-995e-ac96039f4014 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:21.882651 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740dbbef-8db4-4d58-995e-ac96039f4014 admin admin] PATH_INFO: `/v3/users/ebb0a9305a9246939961e643642f52b5` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:21.885881 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740dbbef-8db4-4d58-995e-ac96039f4014 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ebb0a9305a9246939961e643642f52b5)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:21.888642 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740dbbef-8db4-4d58-995e-ac96039f4014 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:21.917459 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:00:22.073943 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-740dbbef-8db4-4d58-995e-ac96039f4014 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ebb0a9305a9246939961e643642f52b5'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:22.114277 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-740dbbef-8db4-4d58-995e-ac96039f4014 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ebb0a9305a9246939961e643642f52b5'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:22.115296 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-740dbbef-8db4-4d58-995e-ac96039f4014 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ebb0a9305a9246939961e643642f52b5'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:22.132136 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-740dbbef-8db4-4d58-995e-ac96039f4014 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ebb0a9305a9246939961e643642f52b5'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:22.133377 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-740dbbef-8db4-4d58-995e-ac96039f4014 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ebb0a9305a9246939961e643642f52b5'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:22.183344 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 739/1481] 10.222.0.59 () {58 vars in 1224 bytes} [Tue Oct 11 12:00:21 2022] DELETE /identity/v3/users/ebb0a9305a9246939961e643642f52b5 => generated 0 bytes in 527 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 12:00:22.184698 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740dbbef-8db4-4d58-995e-ac96039f4014 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:22.188013 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-740dbbef-8db4-4d58-995e-ac96039f4014 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:22.204011 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 743/1482] 10.222.0.59 () {58 vars in 1009 bytes} [Tue Oct 11 12:00:21 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 496 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 12:00:22.441270 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740dbbef-8db4-4d58-995e-ac96039f4014 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:22.462339 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b8fad4-a84b-4042-a0ca-7cbe304a851d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:22.473395 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b8fad4-a84b-4042-a0ca-7cbe304a851d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:22.473706 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b8fad4-a84b-4042-a0ca-7cbe304a851d admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:22.474692 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b8fad4-a84b-4042-a0ca-7cbe304a851d admin admin] PATH_INFO: `/v3/projects/14d8e0913e38452c896f11531145f961` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:22.481444 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b8fad4-a84b-4042-a0ca-7cbe304a851d admin admin] RBAC: Authorizing `identity:delete_project(project_id=14d8e0913e38452c896f11531145f961)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:22.491948 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b8fad4-a84b-4042-a0ca-7cbe304a851d admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:22.510722 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:00:22.580714 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e61403e-78b7-4e97-9b20-181a6db99829 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:22.582657 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-6e61403e-78b7-4e97-9b20-181a6db99829 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:22.695298 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-65b8fad4-a84b-4042-a0ca-7cbe304a851d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14d8e0913e38452c896f11531145f961', 'request_id': 'req-65b8fad4-a84b-4042-a0ca-7cbe304a851d'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:22.728167 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-65b8fad4-a84b-4042-a0ca-7cbe304a851d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14d8e0913e38452c896f11531145f961', 'request_id': 'req-65b8fad4-a84b-4042-a0ca-7cbe304a851d'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:22.753544 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-65b8fad4-a84b-4042-a0ca-7cbe304a851d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14d8e0913e38452c896f11531145f961', 'request_id': 'req-65b8fad4-a84b-4042-a0ca-7cbe304a851d'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:22.800025 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-65b8fad4-a84b-4042-a0ca-7cbe304a851d admin admin] The token cache is being invalidate because project 14d8e0913e38452c896f11531145f961 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81432) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 12:00:22.800388 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-65b8fad4-a84b-4042-a0ca-7cbe304a851d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:22.809486 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-65b8fad4-a84b-4042-a0ca-7cbe304a851d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:22.819546 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 740/1483] 10.222.0.59 () {58 vars in 1233 bytes} [Tue Oct 11 12:00:22 2022] DELETE /identity/v3/projects/14d8e0913e38452c896f11531145f961 => generated 0 bytes in 632 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:00:22.821233 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b8fad4-a84b-4042-a0ca-7cbe304a851d admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:22.829497 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-65b8fad4-a84b-4042-a0ca-7cbe304a851d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:23.003901 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b8fad4-a84b-4042-a0ca-7cbe304a851d admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:23.018032 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5d866c-ef96-4431-852c-2f212c9f3caf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:23.018769 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5d866c-ef96-4431-852c-2f212c9f3caf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:23.019097 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5d866c-ef96-4431-852c-2f212c9f3caf admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:23.019396 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5d866c-ef96-4431-852c-2f212c9f3caf admin admin] PATH_INFO: `/v3/users/abd03b717caa41dc953f6874335b7551` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:23.025905 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5d866c-ef96-4431-852c-2f212c9f3caf admin admin] RBAC: Authorizing `identity:delete_user(user_id=abd03b717caa41dc953f6874335b7551)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:23.031149 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5d866c-ef96-4431-852c-2f212c9f3caf admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:23.105349 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e61403e-78b7-4e97-9b20-181a6db99829 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:23.171172 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-da5d866c-ef96-4431-852c-2f212c9f3caf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'abd03b717caa41dc953f6874335b7551'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:23.198691 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-da5d866c-ef96-4431-852c-2f212c9f3caf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'abd03b717caa41dc953f6874335b7551'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:23.203613 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-da5d866c-ef96-4431-852c-2f212c9f3caf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'abd03b717caa41dc953f6874335b7551'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:23.225006 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-da5d866c-ef96-4431-852c-2f212c9f3caf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'abd03b717caa41dc953f6874335b7551'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:23.225803 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-da5d866c-ef96-4431-852c-2f212c9f3caf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'abd03b717caa41dc953f6874335b7551'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:23.276414 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 741/1484] 10.222.0.59 () {58 vars in 1224 bytes} [Tue Oct 11 12:00:22 2022] DELETE /identity/v3/users/abd03b717caa41dc953f6874335b7551 => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:00:23.346007 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a668438-34b6-40da-b1b7-82d83ca72d31 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76e7fee1993a40d5ba4295e7daeb3e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:23.349270 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a668438-34b6-40da-b1b7-82d83ca72d31 service swift] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:23.350064 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a668438-34b6-40da-b1b7-82d83ca72d31 service swift] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:23.351190 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a668438-34b6-40da-b1b7-82d83ca72d31 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:23.356639 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-5a668438-34b6-40da-b1b7-82d83ca72d31 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:23.448922 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5d866c-ef96-4431-852c-2f212c9f3caf admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:23.452266 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-da5d866c-ef96-4431-852c-2f212c9f3caf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:23.495662 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:00:23.562008 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:00:23.614838 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a668438-34b6-40da-b1b7-82d83ca72d31 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:23.618587 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a668438-34b6-40da-b1b7-82d83ca72d31 service swift] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:23.652666 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 744/1485] 10.222.0.59 () {60 vars in 1384 bytes} [Tue Oct 11 12:00:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 1072 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 11 12:00:23.658319 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a668438-34b6-40da-b1b7-82d83ca72d31 service swift] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:23.684907 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-5a668438-34b6-40da-b1b7-82d83ca72d31 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:23.760327 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a668438-34b6-40da-b1b7-82d83ca72d31 service swift] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:23.761544 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5d866c-ef96-4431-852c-2f212c9f3caf admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:23.769738 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f796bbb8-266e-4865-9421-d98344b411fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:23.772025 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f796bbb8-266e-4865-9421-d98344b411fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:23.772850 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f796bbb8-266e-4865-9421-d98344b411fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:23.773325 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f796bbb8-266e-4865-9421-d98344b411fa admin admin] PATH_INFO: `/v3/users/2b616e9828314e95937990d0a9278c60` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:23.774586 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f203c2b1-c7d0-4338-a85f-6fa73c720a72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:23.775591 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f203c2b1-c7d0-4338-a85f-6fa73c720a72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:23.775900 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f203c2b1-c7d0-4338-a85f-6fa73c720a72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:23.776366 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f203c2b1-c7d0-4338-a85f-6fa73c720a72 admin admin] PATH_INFO: `/v3/users/456d8be3a30840a19a3cdb8b82662bff` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:23.776772 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f796bbb8-266e-4865-9421-d98344b411fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b616e9828314e95937990d0a9278c60)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:23.778202 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f203c2b1-c7d0-4338-a85f-6fa73c720a72 admin admin] RBAC: Authorizing `identity:delete_user(user_id=456d8be3a30840a19a3cdb8b82662bff)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:23.780706 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f203c2b1-c7d0-4338-a85f-6fa73c720a72 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:23.781423 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f796bbb8-266e-4865-9421-d98344b411fa admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:23.916022 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-f203c2b1-c7d0-4338-a85f-6fa73c720a72 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '456d8be3a30840a19a3cdb8b82662bff'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:23.957075 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-f796bbb8-266e-4865-9421-d98344b411fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b616e9828314e95937990d0a9278c60'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:23.993914 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-f796bbb8-266e-4865-9421-d98344b411fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b616e9828314e95937990d0a9278c60'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:23.998190 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-f203c2b1-c7d0-4338-a85f-6fa73c720a72 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '456d8be3a30840a19a3cdb8b82662bff'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:23.999645 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-f203c2b1-c7d0-4338-a85f-6fa73c720a72 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '456d8be3a30840a19a3cdb8b82662bff'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:24.007838 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-f796bbb8-266e-4865-9421-d98344b411fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b616e9828314e95937990d0a9278c60'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:24.009096 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-f796bbb8-266e-4865-9421-d98344b411fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b616e9828314e95937990d0a9278c60'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:24.025015 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-f796bbb8-266e-4865-9421-d98344b411fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b616e9828314e95937990d0a9278c60'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:24.029981 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-f203c2b1-c7d0-4338-a85f-6fa73c720a72 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '456d8be3a30840a19a3cdb8b82662bff'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:24.031009 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-f203c2b1-c7d0-4338-a85f-6fa73c720a72 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '456d8be3a30840a19a3cdb8b82662bff'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:24.056745 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 745/1486] 10.222.0.59 () {58 vars in 1224 bytes} [Tue Oct 11 12:00:23 2022] DELETE /identity/v3/users/2b616e9828314e95937990d0a9278c60 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:00:24.061967 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6470af61-44bb-4671-9145-3060bf80ba4b None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:24.067004 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 742/1487] 10.222.0.59 () {58 vars in 1224 bytes} [Tue Oct 11 12:00:23 2022] DELETE /identity/v3/users/456d8be3a30840a19a3cdb8b82662bff => generated 0 bytes in 614 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:00:24.068077 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86cb53d-67ee-4a70-9040-724356272d91 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:24.068375 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86cb53d-67ee-4a70-9040-724356272d91 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:24.069241 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6470af61-44bb-4671-9145-3060bf80ba4b None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:24.070302 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6470af61-44bb-4671-9145-3060bf80ba4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:24.072799 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86cb53d-67ee-4a70-9040-724356272d91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:24.105052 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-6470af61-44bb-4671-9145-3060bf80ba4b None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:00:24.112661 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-6470af61-44bb-4671-9145-3060bf80ba4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:24.134124 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-c86cb53d-67ee-4a70-9040-724356272d91 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:00:24.139641 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-c86cb53d-67ee-4a70-9040-724356272d91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:24.342682 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 746/1488] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:00:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:00:24.353178 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 743/1489] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:00:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:00:24.381455 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6470af61-44bb-4671-9145-3060bf80ba4b None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:24.383514 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-6470af61-44bb-4671-9145-3060bf80ba4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:24.388430 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86cb53d-67ee-4a70-9040-724356272d91 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:24.390269 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-c86cb53d-67ee-4a70-9040-724356272d91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:24.460911 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:00:24.518914 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86cb53d-67ee-4a70-9040-724356272d91 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:24.521428 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6470af61-44bb-4671-9145-3060bf80ba4b None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:24.523812 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f269eb9-4d47-47a8-8fef-34792ae7e325 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e563fe174db4cc19ae6bf8162cc3d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:24.524487 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f269eb9-4d47-47a8-8fef-34792ae7e325 service neutron] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:24.524767 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f269eb9-4d47-47a8-8fef-34792ae7e325 service neutron] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:24.525131 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f269eb9-4d47-47a8-8fef-34792ae7e325 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:24.526749 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3db12b8-ec76-426e-b229-2267d1312908 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e563fe174db4cc19ae6bf8162cc3d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:24.527584 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3db12b8-ec76-426e-b229-2267d1312908 service neutron] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:24.528434 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3db12b8-ec76-426e-b229-2267d1312908 service neutron] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:24.530460 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3db12b8-ec76-426e-b229-2267d1312908 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:24.550733 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3db12b8-ec76-426e-b229-2267d1312908 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:24.554761 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3db12b8-ec76-426e-b229-2267d1312908 service neutron] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:24.566924 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f269eb9-4d47-47a8-8fef-34792ae7e325 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:24.568695 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f269eb9-4d47-47a8-8fef-34792ae7e325 service neutron] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:24.593251 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:00:24.597662 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 744/1490] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:00:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 11 12:00:24.598723 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f269eb9-4d47-47a8-8fef-34792ae7e325 service neutron] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:24.600185 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 747/1491] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:00:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:00:24.600578 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-8f269eb9-4d47-47a8-8fef-34792ae7e325 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:24.616886 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f269eb9-4d47-47a8-8fef-34792ae7e325 service neutron] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:24.624697 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be53a79c-71e2-4e7e-9bac-a2aeb4ae2f2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:24.625489 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be53a79c-71e2-4e7e-9bac-a2aeb4ae2f2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:24.626146 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be53a79c-71e2-4e7e-9bac-a2aeb4ae2f2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:24.626407 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be53a79c-71e2-4e7e-9bac-a2aeb4ae2f2b admin admin] PATH_INFO: `/v3/projects/2fa89b22c474403686edde15d32c553e` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:24.629100 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be53a79c-71e2-4e7e-9bac-a2aeb4ae2f2b admin admin] RBAC: Authorizing `identity:delete_project(project_id=2fa89b22c474403686edde15d32c553e)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:24.631217 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be53a79c-71e2-4e7e-9bac-a2aeb4ae2f2b admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:24.727307 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-be53a79c-71e2-4e7e-9bac-a2aeb4ae2f2b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2fa89b22c474403686edde15d32c553e', 'request_id': 'req-be53a79c-71e2-4e7e-9bac-a2aeb4ae2f2b'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:24.735390 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-be53a79c-71e2-4e7e-9bac-a2aeb4ae2f2b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2fa89b22c474403686edde15d32c553e', 'request_id': 'req-be53a79c-71e2-4e7e-9bac-a2aeb4ae2f2b'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:24.755688 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-be53a79c-71e2-4e7e-9bac-a2aeb4ae2f2b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2fa89b22c474403686edde15d32c553e', 'request_id': 'req-be53a79c-71e2-4e7e-9bac-a2aeb4ae2f2b'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:24.774764 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-be53a79c-71e2-4e7e-9bac-a2aeb4ae2f2b admin admin] The token cache is being invalidate because project 2fa89b22c474403686edde15d32c553e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81432) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 12:00:24.776087 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-be53a79c-71e2-4e7e-9bac-a2aeb4ae2f2b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:24.777045 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-be53a79c-71e2-4e7e-9bac-a2aeb4ae2f2b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:24.779256 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 745/1492] 10.222.0.59 () {58 vars in 1233 bytes} [Tue Oct 11 12:00:24 2022] DELETE /identity/v3/projects/2fa89b22c474403686edde15d32c553e => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:00:24.786848 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1abcef-6c53-4d20-b5d7-b8e92b2f95b1 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:24.787142 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1abcef-6c53-4d20-b5d7-b8e92b2f95b1 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:24.787423 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1abcef-6c53-4d20-b5d7-b8e92b2f95b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:24.817967 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-ba1abcef-6c53-4d20-b5d7-b8e92b2f95b1 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:00:24.819720 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-ba1abcef-6c53-4d20-b5d7-b8e92b2f95b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:24.982135 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 748/1493] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:00:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:00:24.983609 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fd0703-82d5-4fdc-945e-c298d2950f1a None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:24.983903 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fd0703-82d5-4fdc-945e-c298d2950f1a None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:24.984168 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fd0703-82d5-4fdc-945e-c298d2950f1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:25.038821 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-71fd0703-82d5-4fdc-945e-c298d2950f1a None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:00:25.041635 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-71fd0703-82d5-4fdc-945e-c298d2950f1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:25.070210 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 746/1494] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:00:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:00:25.076841 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1abcef-6c53-4d20-b5d7-b8e92b2f95b1 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:25.093190 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1abcef-6c53-4d20-b5d7-b8e92b2f95b1 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:25.113267 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078b8d82-9a79-4a66-8912-92bd0f5af20f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:25.114110 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078b8d82-9a79-4a66-8912-92bd0f5af20f admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:25.114439 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078b8d82-9a79-4a66-8912-92bd0f5af20f admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:25.115673 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078b8d82-9a79-4a66-8912-92bd0f5af20f admin admin] PATH_INFO: `/v3/domains` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:25.117660 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078b8d82-9a79-4a66-8912-92bd0f5af20f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 12:00:25.117970 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078b8d82-9a79-4a66-8912-92bd0f5af20f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:25.120118 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078b8d82-9a79-4a66-8912-92bd0f5af20f admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:25.158684 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 749/1495] 10.222.0.59 () {58 vars in 1153 bytes} [Tue Oct 11 12:00:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 12:00:25.163284 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fd0703-82d5-4fdc-945e-c298d2950f1a None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:25.194943 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fd0703-82d5-4fdc-945e-c298d2950f1a None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:25.205438 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7925beca-edc9-453c-813a-82e92bec652b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:25.207484 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7925beca-edc9-453c-813a-82e92bec652b admin admin] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:25.207985 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7925beca-edc9-453c-813a-82e92bec652b admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:25.208505 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7925beca-edc9-453c-813a-82e92bec652b admin admin] PATH_INFO: `/v3/projects` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:25.210054 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7925beca-edc9-453c-813a-82e92bec652b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:25.214374 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7925beca-edc9-453c-813a-82e92bec652b admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:25.280076 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 747/1496] 10.222.0.59 () {60 vars in 1153 bytes} [Tue Oct 11 12:00:25 2022] POST /identity/v3/projects => generated 373 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:00:25.287992 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078b8d82-9a79-4a66-8912-92bd0f5af20f admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:25.310324 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078b8d82-9a79-4a66-8912-92bd0f5af20f admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:25.454642 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8458352f-1bf8-411e-b61c-153086401c73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:25.455751 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8458352f-1bf8-411e-b61c-153086401c73 admin admin] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:25.456251 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8458352f-1bf8-411e-b61c-153086401c73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:25.456793 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8458352f-1bf8-411e-b61c-153086401c73 admin admin] PATH_INFO: `/v3/users` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:25.458055 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8458352f-1bf8-411e-b61c-153086401c73 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:25.461914 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8458352f-1bf8-411e-b61c-153086401c73 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:25.511873 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 750/1497] 10.222.0.59 () {60 vars in 1144 bytes} [Tue Oct 11 12:00:25 2022] POST /identity/v3/users => generated 287 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 12:00:25.513642 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7925beca-edc9-453c-813a-82e92bec652b admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:25.542223 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7925beca-edc9-453c-813a-82e92bec652b admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:25.554243 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ecb6b1-a8d4-42b2-8eae-4d32d9f7ba8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:25.556017 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ecb6b1-a8d4-42b2-8eae-4d32d9f7ba8a admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:25.556714 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ecb6b1-a8d4-42b2-8eae-4d32d9f7ba8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:25.557381 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ecb6b1-a8d4-42b2-8eae-4d32d9f7ba8a admin admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:25.557737 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8458352f-1bf8-411e-b61c-153086401c73 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:25.559357 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ecb6b1-a8d4-42b2-8eae-4d32d9f7ba8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:25.561110 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-8458352f-1bf8-411e-b61c-153086401c73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:25.563092 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ecb6b1-a8d4-42b2-8eae-4d32d9f7ba8a admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:25.594699 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 748/1498] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:00:25 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:00:25.596894 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ecb6b1-a8d4-42b2-8eae-4d32d9f7ba8a admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:25.604777 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8458352f-1bf8-411e-b61c-153086401c73 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:25.616171 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96da647-c705-4182-b2c1-dbdd5967776a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:25.616944 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96da647-c705-4182-b2c1-dbdd5967776a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:25.617233 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96da647-c705-4182-b2c1-dbdd5967776a admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:25.617564 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96da647-c705-4182-b2c1-dbdd5967776a admin admin] PATH_INFO: `/v3/projects/c6a60db4ccdd4ef7a10e481abc983575` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:25.619084 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96da647-c705-4182-b2c1-dbdd5967776a admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6a60db4ccdd4ef7a10e481abc983575)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:25.621261 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96da647-c705-4182-b2c1-dbdd5967776a admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:25.621502 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ecb6b1-a8d4-42b2-8eae-4d32d9f7ba8a admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:25.659608 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4f6554-cdb2-4ceb-92a4-e41155a24aa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:25.663864 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4f6554-cdb2-4ceb-92a4-e41155a24aa2 admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:25.664491 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4f6554-cdb2-4ceb-92a4-e41155a24aa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:25.666485 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4f6554-cdb2-4ceb-92a4-e41155a24aa2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:25.668807 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4f6554-cdb2-4ceb-92a4-e41155a24aa2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:25.672926 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4f6554-cdb2-4ceb-92a4-e41155a24aa2 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:25.710204 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 749/1499] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:00:25 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:00:25.712868 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4f6554-cdb2-4ceb-92a4-e41155a24aa2 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:25.741862 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-4c4f6554-cdb2-4ceb-92a4-e41155a24aa2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:25.802673 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4f6554-cdb2-4ceb-92a4-e41155a24aa2 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:25.831034 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-e96da647-c705-4182-b2c1-dbdd5967776a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6a60db4ccdd4ef7a10e481abc983575', 'request_id': 'req-e96da647-c705-4182-b2c1-dbdd5967776a'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:25.850994 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-e96da647-c705-4182-b2c1-dbdd5967776a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6a60db4ccdd4ef7a10e481abc983575', 'request_id': 'req-e96da647-c705-4182-b2c1-dbdd5967776a'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:25.865430 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-e96da647-c705-4182-b2c1-dbdd5967776a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6a60db4ccdd4ef7a10e481abc983575', 'request_id': 'req-e96da647-c705-4182-b2c1-dbdd5967776a'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:25.876927 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:00:25.897728 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-e96da647-c705-4182-b2c1-dbdd5967776a admin admin] The token cache is being invalidate because project c6a60db4ccdd4ef7a10e481abc983575 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 12:00:25.898133 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-e96da647-c705-4182-b2c1-dbdd5967776a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:25.899349 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-e96da647-c705-4182-b2c1-dbdd5967776a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:25.901581 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 751/1500] 10.222.0.59 () {58 vars in 1233 bytes} [Tue Oct 11 12:00:25 2022] DELETE /identity/v3/projects/c6a60db4ccdd4ef7a10e481abc983575 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 12:00:25.903043 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96da647-c705-4182-b2c1-dbdd5967776a admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:25.905179 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-e96da647-c705-4182-b2c1-dbdd5967776a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:25.939879 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee4c9c5-8eed-446f-befd-830ae0570bd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:25.941245 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee4c9c5-8eed-446f-befd-830ae0570bd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:25.941693 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee4c9c5-8eed-446f-befd-830ae0570bd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:25.942126 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee4c9c5-8eed-446f-befd-830ae0570bd1 admin admin] PATH_INFO: `/v3/projects/c77e600b1b3746aeb4468fb01886b74d` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:25.943667 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee4c9c5-8eed-446f-befd-830ae0570bd1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c77e600b1b3746aeb4468fb01886b74d)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:25.946140 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee4c9c5-8eed-446f-befd-830ae0570bd1 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:25.967689 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:00:26.055523 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-0ee4c9c5-8eed-446f-befd-830ae0570bd1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c77e600b1b3746aeb4468fb01886b74d', 'request_id': 'req-0ee4c9c5-8eed-446f-befd-830ae0570bd1'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:26.062846 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96da647-c705-4182-b2c1-dbdd5967776a admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:26.065598 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-0ee4c9c5-8eed-446f-befd-830ae0570bd1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c77e600b1b3746aeb4468fb01886b74d', 'request_id': 'req-0ee4c9c5-8eed-446f-befd-830ae0570bd1'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:26.076020 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-0ee4c9c5-8eed-446f-befd-830ae0570bd1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c77e600b1b3746aeb4468fb01886b74d', 'request_id': 'req-0ee4c9c5-8eed-446f-befd-830ae0570bd1'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:26.096978 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-0ee4c9c5-8eed-446f-befd-830ae0570bd1 admin admin] The token cache is being invalidate because project c77e600b1b3746aeb4468fb01886b74d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81432) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 12:00:26.097938 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-0ee4c9c5-8eed-446f-befd-830ae0570bd1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:26.099189 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-0ee4c9c5-8eed-446f-befd-830ae0570bd1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:26.114842 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 750/1501] 10.222.0.59 () {58 vars in 1233 bytes} [Tue Oct 11 12:00:25 2022] DELETE /identity/v3/projects/c77e600b1b3746aeb4468fb01886b74d => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:00:26.114842 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee4c9c5-8eed-446f-befd-830ae0570bd1 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:26.114842 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-0ee4c9c5-8eed-446f-befd-830ae0570bd1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:26.153270 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c957e2-5e2b-444c-96e8-73762f4f95c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:26.154050 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c957e2-5e2b-444c-96e8-73762f4f95c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:26.154358 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c957e2-5e2b-444c-96e8-73762f4f95c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:26.154708 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c957e2-5e2b-444c-96e8-73762f4f95c3 admin admin] PATH_INFO: `/v3/projects/0b1ac274b4bb4997a54db5eebd03627d/users/3f91177e52734ab79932d8ca6603b7c4/roles/de2d2bd26fff4748a79990f97a107164` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:26.171855 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c957e2-5e2b-444c-96e8-73762f4f95c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b1ac274b4bb4997a54db5eebd03627d, user_id=3f91177e52734ab79932d8ca6603b7c4, role_id=de2d2bd26fff4748a79990f97a107164)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:26.175286 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c957e2-5e2b-444c-96e8-73762f4f95c3 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:26.201262 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 752/1502] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 12:00:25 2022] PUT /identity/v3/projects/0b1ac274b4bb4997a54db5eebd03627d/users/3f91177e52734ab79932d8ca6603b7c4/roles/de2d2bd26fff4748a79990f97a107164 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:00:26.202517 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983f38c8-4612-4a34-a8fd-16181a82c853 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:26.202926 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983f38c8-4612-4a34-a8fd-16181a82c853 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:26.203087 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983f38c8-4612-4a34-a8fd-16181a82c853 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:26.232859 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-983f38c8-4612-4a34-a8fd-16181a82c853 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:00:26.234274 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-983f38c8-4612-4a34-a8fd-16181a82c853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:26.244975 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee4c9c5-8eed-446f-befd-830ae0570bd1 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:26.334237 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8b8045-8b3a-4e49-b405-4cd25711141c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:26.335411 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8b8045-8b3a-4e49-b405-4cd25711141c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:26.335890 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8b8045-8b3a-4e49-b405-4cd25711141c admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:26.336383 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8b8045-8b3a-4e49-b405-4cd25711141c admin admin] PATH_INFO: `/v3/projects/85a10ff66bc5457899b9abae9b27b903` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:26.338426 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8b8045-8b3a-4e49-b405-4cd25711141c admin admin] RBAC: Authorizing `identity:delete_project(project_id=85a10ff66bc5457899b9abae9b27b903)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:26.342126 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8b8045-8b3a-4e49-b405-4cd25711141c admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:26.353809 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 753/1503] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:00:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:00:26.353809 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983f38c8-4612-4a34-a8fd-16181a82c853 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:26.356191 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-983f38c8-4612-4a34-a8fd-16181a82c853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:26.377910 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983f38c8-4612-4a34-a8fd-16181a82c853 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:26.387552 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983f9d7c-a7e9-4ba7-bc6e-5ad500897dac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:26.389042 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983f9d7c-a7e9-4ba7-bc6e-5ad500897dac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:26.389531 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983f9d7c-a7e9-4ba7-bc6e-5ad500897dac admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:26.390288 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983f9d7c-a7e9-4ba7-bc6e-5ad500897dac admin admin] PATH_INFO: `/v3/users/76d6e00fb4c84488a0cc0846c0e8bdbc` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:26.392720 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983f9d7c-a7e9-4ba7-bc6e-5ad500897dac admin admin] RBAC: Authorizing `identity:delete_user(user_id=76d6e00fb4c84488a0cc0846c0e8bdbc)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:26.396661 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983f9d7c-a7e9-4ba7-bc6e-5ad500897dac admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:26.428155 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:00:26.431905 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:00:26.432787 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-bf8b8045-8b3a-4e49-b405-4cd25711141c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85a10ff66bc5457899b9abae9b27b903', 'request_id': 'req-bf8b8045-8b3a-4e49-b405-4cd25711141c'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:26.443289 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-bf8b8045-8b3a-4e49-b405-4cd25711141c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85a10ff66bc5457899b9abae9b27b903', 'request_id': 'req-bf8b8045-8b3a-4e49-b405-4cd25711141c'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:26.455338 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-bf8b8045-8b3a-4e49-b405-4cd25711141c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85a10ff66bc5457899b9abae9b27b903', 'request_id': 'req-bf8b8045-8b3a-4e49-b405-4cd25711141c'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:26.466214 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-983f9d7c-a7e9-4ba7-bc6e-5ad500897dac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76d6e00fb4c84488a0cc0846c0e8bdbc'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:26.480023 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-bf8b8045-8b3a-4e49-b405-4cd25711141c admin admin] The token cache is being invalidate because project 85a10ff66bc5457899b9abae9b27b903 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81432) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 12:00:26.480536 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-bf8b8045-8b3a-4e49-b405-4cd25711141c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:26.481731 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-bf8b8045-8b3a-4e49-b405-4cd25711141c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:26.484129 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-983f9d7c-a7e9-4ba7-bc6e-5ad500897dac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76d6e00fb4c84488a0cc0846c0e8bdbc'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:26.484767 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 751/1504] 10.222.0.59 () {58 vars in 1233 bytes} [Tue Oct 11 12:00:26 2022] DELETE /identity/v3/projects/85a10ff66bc5457899b9abae9b27b903 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:00:26.486899 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296a3e19-20e2-40c5-81f8-26a4707a5a20 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:26.487376 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296a3e19-20e2-40c5-81f8-26a4707a5a20 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:26.487922 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296a3e19-20e2-40c5-81f8-26a4707a5a20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:26.496947 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-983f9d7c-a7e9-4ba7-bc6e-5ad500897dac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76d6e00fb4c84488a0cc0846c0e8bdbc'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:26.497704 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-983f9d7c-a7e9-4ba7-bc6e-5ad500897dac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76d6e00fb4c84488a0cc0846c0e8bdbc'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:26.507553 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-983f9d7c-a7e9-4ba7-bc6e-5ad500897dac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76d6e00fb4c84488a0cc0846c0e8bdbc'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:26.515391 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 754/1505] 10.222.0.59 () {58 vars in 1224 bytes} [Tue Oct 11 12:00:26 2022] DELETE /identity/v3/users/76d6e00fb4c84488a0cc0846c0e8bdbc => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:00:26.516802 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c992a4d1-37da-4e04-b352-72275607d83f None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:26.517155 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c992a4d1-37da-4e04-b352-72275607d83f None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:26.517397 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c992a4d1-37da-4e04-b352-72275607d83f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:26.541563 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-c992a4d1-37da-4e04-b352-72275607d83f None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:00:26.543133 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-c992a4d1-37da-4e04-b352-72275607d83f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:26.549924 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-296a3e19-20e2-40c5-81f8-26a4707a5a20 None None] MFA Rules not processed for user `3f91177e52734ab79932d8ca6603b7c4`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:00:26.552655 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-296a3e19-20e2-40c5-81f8-26a4707a5a20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:26.651567 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 755/1506] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:00:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:00:26.653354 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c7afe0-3af3-4983-a4a9-2d7f202ba5c0 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:26.653776 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c7afe0-3af3-4983-a4a9-2d7f202ba5c0 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:26.653945 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c7afe0-3af3-4983-a4a9-2d7f202ba5c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:26.689068 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 752/1507] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:00:26 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:00:26.690882 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296a3e19-20e2-40c5-81f8-26a4707a5a20 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:26.697191 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-44c7afe0-3af3-4983-a4a9-2d7f202ba5c0 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:00:26.699567 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-44c7afe0-3af3-4983-a4a9-2d7f202ba5c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:26.709974 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296a3e19-20e2-40c5-81f8-26a4707a5a20 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:26.715101 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 756/1508] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:00:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:00:26.716586 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c7afe0-3af3-4983-a4a9-2d7f202ba5c0 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:26.718803 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-44c7afe0-3af3-4983-a4a9-2d7f202ba5c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:26.722775 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188a67d8-64ca-4a39-b270-6d99583af5ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:26.723614 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188a67d8-64ca-4a39-b270-6d99583af5ff admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:26.723822 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188a67d8-64ca-4a39-b270-6d99583af5ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:26.724346 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188a67d8-64ca-4a39-b270-6d99583af5ff admin admin] PATH_INFO: `/v3/domains` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:26.724990 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188a67d8-64ca-4a39-b270-6d99583af5ff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 12:00:26.725284 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188a67d8-64ca-4a39-b270-6d99583af5ff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:26.727386 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188a67d8-64ca-4a39-b270-6d99583af5ff admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:26.744375 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c7afe0-3af3-4983-a4a9-2d7f202ba5c0 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:26.747941 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 753/1509] 10.222.0.59 () {58 vars in 1153 bytes} [Tue Oct 11 12:00:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 12:00:26.749607 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a1418e-c04b-4e68-8ed4-61a5a7bfcd5c None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:26.750302 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a1418e-c04b-4e68-8ed4-61a5a7bfcd5c None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:26.750481 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a1418e-c04b-4e68-8ed4-61a5a7bfcd5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:26.753578 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98887dad-7c84-49d2-a2b4-9ffc8fc26504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:26.754723 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98887dad-7c84-49d2-a2b4-9ffc8fc26504 admin admin] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:26.755198 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98887dad-7c84-49d2-a2b4-9ffc8fc26504 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:26.755701 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98887dad-7c84-49d2-a2b4-9ffc8fc26504 admin admin] PATH_INFO: `/v3/projects` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:26.756870 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98887dad-7c84-49d2-a2b4-9ffc8fc26504 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:26.761033 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98887dad-7c84-49d2-a2b4-9ffc8fc26504 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:26.773810 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-86a1418e-c04b-4e68-8ed4-61a5a7bfcd5c None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:00:26.775247 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-86a1418e-c04b-4e68-8ed4-61a5a7bfcd5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:26.786208 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 754/1510] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:00:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:00:26.788911 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a1418e-c04b-4e68-8ed4-61a5a7bfcd5c None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:26.808662 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a1418e-c04b-4e68-8ed4-61a5a7bfcd5c None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:26.827236 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250a2db7-7260-4d3c-8638-fff31ab728c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:26.828402 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250a2db7-7260-4d3c-8638-fff31ab728c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:26.829329 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250a2db7-7260-4d3c-8638-fff31ab728c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:26.829434 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250a2db7-7260-4d3c-8638-fff31ab728c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:26.830547 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 757/1511] 10.222.0.59 () {60 vars in 1153 bytes} [Tue Oct 11 12:00:26 2022] POST /identity/v3/projects => generated 373 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:00:26.830711 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250a2db7-7260-4d3c-8638-fff31ab728c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:26.834821 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250a2db7-7260-4d3c-8638-fff31ab728c0 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:26.835030 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98887dad-7c84-49d2-a2b4-9ffc8fc26504 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:26.912645 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 755/1512] 10.222.0.59 () {60 vars in 1153 bytes} [Tue Oct 11 12:00:26 2022] POST /identity/v3/projects => generated 371 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:00:26.914826 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250a2db7-7260-4d3c-8638-fff31ab728c0 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:27.000163 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98887dad-7c84-49d2-a2b4-9ffc8fc26504 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:27.025942 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:DELETE=6,keystone:SELECT=92,keystone:INSERT=2 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:00:27.072677 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250a2db7-7260-4d3c-8638-fff31ab728c0 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:27.082399 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a376e28-719e-453f-937b-9f628dccd616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:27.085168 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a376e28-719e-453f-937b-9f628dccd616 admin admin] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:27.085759 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a376e28-719e-453f-937b-9f628dccd616 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:27.086160 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a376e28-719e-453f-937b-9f628dccd616 admin admin] PATH_INFO: `/v3/users` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:27.087412 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a376e28-719e-453f-937b-9f628dccd616 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:27.090890 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3eeb01-1538-4b9d-b7d7-f69a5f37b41b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:27.091648 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3eeb01-1538-4b9d-b7d7-f69a5f37b41b admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:27.091939 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3eeb01-1538-4b9d-b7d7-f69a5f37b41b admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:27.092288 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3eeb01-1538-4b9d-b7d7-f69a5f37b41b admin admin] PATH_INFO: `/v3/domains` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:27.093231 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3eeb01-1538-4b9d-b7d7-f69a5f37b41b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 12:00:27.093231 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3eeb01-1538-4b9d-b7d7-f69a5f37b41b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:27.094876 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3eeb01-1538-4b9d-b7d7-f69a5f37b41b admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:27.096268 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a376e28-719e-453f-937b-9f628dccd616 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:27.126730 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 758/1513] 10.222.0.59 () {58 vars in 1153 bytes} [Tue Oct 11 12:00:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 12:00:27.132264 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3eeb01-1538-4b9d-b7d7-f69a5f37b41b admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:27.133296 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 756/1514] 10.222.0.59 () {60 vars in 1144 bytes} [Tue Oct 11 12:00:26 2022] POST /identity/v3/users => generated 293 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:00:27.135769 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a376e28-719e-453f-937b-9f628dccd616 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:27.138884 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-6a376e28-719e-453f-937b-9f628dccd616 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:27.151967 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3eeb01-1538-4b9d-b7d7-f69a5f37b41b admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:27.160902 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a31d6bc-26e4-4f76-bb3b-c115e2f92f70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:27.162292 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a31d6bc-26e4-4f76-bb3b-c115e2f92f70 admin admin] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:27.162848 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a31d6bc-26e4-4f76-bb3b-c115e2f92f70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:27.163355 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a31d6bc-26e4-4f76-bb3b-c115e2f92f70 admin admin] PATH_INFO: `/v3/users` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:27.164606 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a31d6bc-26e4-4f76-bb3b-c115e2f92f70 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:27.168462 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a31d6bc-26e4-4f76-bb3b-c115e2f92f70 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:27.168711 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a376e28-719e-453f-937b-9f628dccd616 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:27.176680 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de129948-1a04-4efa-a5ed-0b23c39f7581 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:27.177742 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de129948-1a04-4efa-a5ed-0b23c39f7581 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:27.178278 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de129948-1a04-4efa-a5ed-0b23c39f7581 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:27.178809 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de129948-1a04-4efa-a5ed-0b23c39f7581 admin admin] PATH_INFO: `/v3/projects/327ec774905a459c894e7842ba67776c` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:27.181184 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de129948-1a04-4efa-a5ed-0b23c39f7581 admin admin] RBAC: Authorizing `identity:delete_project(project_id=327ec774905a459c894e7842ba67776c)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:27.185921 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de129948-1a04-4efa-a5ed-0b23c39f7581 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:27.201652 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:DELETE=4,keystone:SELECT=91,keystone:INSERT=5 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:00:27.212470 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 759/1515] 10.222.0.59 () {60 vars in 1144 bytes} [Tue Oct 11 12:00:27 2022] POST /identity/v3/users => generated 286 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:00:27.212470 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a31d6bc-26e4-4f76-bb3b-c115e2f92f70 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:27.231658 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a31d6bc-26e4-4f76-bb3b-c115e2f92f70 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:27.239426 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a32da8-4584-4e2b-8a2e-9404d2181b12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:27.240478 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a32da8-4584-4e2b-8a2e-9404d2181b12 admin admin] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:27.241032 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a32da8-4584-4e2b-8a2e-9404d2181b12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:27.241561 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a32da8-4584-4e2b-8a2e-9404d2181b12 admin admin] PATH_INFO: `/v3/projects` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:27.242691 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a32da8-4584-4e2b-8a2e-9404d2181b12 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:27.246033 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a32da8-4584-4e2b-8a2e-9404d2181b12 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:27.268435 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-de129948-1a04-4efa-a5ed-0b23c39f7581 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '327ec774905a459c894e7842ba67776c', 'request_id': 'req-de129948-1a04-4efa-a5ed-0b23c39f7581'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:27.284625 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-de129948-1a04-4efa-a5ed-0b23c39f7581 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '327ec774905a459c894e7842ba67776c', 'request_id': 'req-de129948-1a04-4efa-a5ed-0b23c39f7581'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:27.298271 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-de129948-1a04-4efa-a5ed-0b23c39f7581 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '327ec774905a459c894e7842ba67776c', 'request_id': 'req-de129948-1a04-4efa-a5ed-0b23c39f7581'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:27.299016 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 760/1516] 10.222.0.59 () {60 vars in 1153 bytes} [Tue Oct 11 12:00:27 2022] POST /identity/v3/projects => generated 365 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:00:27.300717 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a32da8-4584-4e2b-8a2e-9404d2181b12 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:27.323601 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-de129948-1a04-4efa-a5ed-0b23c39f7581 admin admin] The token cache is being invalidate because project 327ec774905a459c894e7842ba67776c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81432) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 12:00:27.324126 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-de129948-1a04-4efa-a5ed-0b23c39f7581 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:27.325463 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-de129948-1a04-4efa-a5ed-0b23c39f7581 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:27.328255 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 757/1517] 10.222.0.59 () {58 vars in 1233 bytes} [Tue Oct 11 12:00:27 2022] DELETE /identity/v3/projects/327ec774905a459c894e7842ba67776c => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:00:27.334458 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de129948-1a04-4efa-a5ed-0b23c39f7581 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:27.337672 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-de129948-1a04-4efa-a5ed-0b23c39f7581 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:27.414619 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a32da8-4584-4e2b-8a2e-9404d2181b12 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:27.421204 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4368be4-6bdf-4bd7-b3a3-27f0fad01f47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:27.421853 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4368be4-6bdf-4bd7-b3a3-27f0fad01f47 admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:27.422201 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4368be4-6bdf-4bd7-b3a3-27f0fad01f47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:27.422538 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4368be4-6bdf-4bd7-b3a3-27f0fad01f47 admin admin] PATH_INFO: `/v3/roles` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:27.423469 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4368be4-6bdf-4bd7-b3a3-27f0fad01f47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:27.425654 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4368be4-6bdf-4bd7-b3a3-27f0fad01f47 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:27.436208 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de129948-1a04-4efa-a5ed-0b23c39f7581 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:27.441042 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 761/1518] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:00:27 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:00:27.442135 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4368be4-6bdf-4bd7-b3a3-27f0fad01f47 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:27.444795 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2699a60b-ba17-4b20-95a7-52d7de09a947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:27.445495 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-e4368be4-6bdf-4bd7-b3a3-27f0fad01f47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:27.445635 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2699a60b-ba17-4b20-95a7-52d7de09a947 admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:27.446002 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2699a60b-ba17-4b20-95a7-52d7de09a947 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:27.446430 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2699a60b-ba17-4b20-95a7-52d7de09a947 admin admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:27.447542 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2699a60b-ba17-4b20-95a7-52d7de09a947 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:27.450586 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2699a60b-ba17-4b20-95a7-52d7de09a947 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:27.465357 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4368be4-6bdf-4bd7-b3a3-27f0fad01f47 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:27.465819 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 758/1519] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:00:27 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:00:27.467765 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2699a60b-ba17-4b20-95a7-52d7de09a947 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:27.470516 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-2699a60b-ba17-4b20-95a7-52d7de09a947 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:27.471664 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941acb03-ef6e-4f49-9a44-68b4797b3b73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:27.472557 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941acb03-ef6e-4f49-9a44-68b4797b3b73 admin admin] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:27.472964 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941acb03-ef6e-4f49-9a44-68b4797b3b73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:27.473117 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941acb03-ef6e-4f49-9a44-68b4797b3b73 admin admin] PATH_INFO: `/v3/users` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:27.473777 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941acb03-ef6e-4f49-9a44-68b4797b3b73 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:27.475804 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941acb03-ef6e-4f49-9a44-68b4797b3b73 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:27.486001 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2699a60b-ba17-4b20-95a7-52d7de09a947 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:27.492204 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8fb87e-1f57-4be2-a5bd-42f5129f65c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:27.493217 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8fb87e-1f57-4be2-a5bd-42f5129f65c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:27.493595 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8fb87e-1f57-4be2-a5bd-42f5129f65c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:27.494071 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8fb87e-1f57-4be2-a5bd-42f5129f65c6 admin admin] PATH_INFO: `/v3/users/d5ade4db7fe943b2b7d6b33b7c7b5872` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:27.496677 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8fb87e-1f57-4be2-a5bd-42f5129f65c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d5ade4db7fe943b2b7d6b33b7c7b5872)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:27.500431 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8fb87e-1f57-4be2-a5bd-42f5129f65c6 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:27.508831 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 762/1520] 10.222.0.59 () {60 vars in 1144 bytes} [Tue Oct 11 12:00:27 2022] POST /identity/v3/users => generated 283 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:00:27.508831 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941acb03-ef6e-4f49-9a44-68b4797b3b73 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:27.509426 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-941acb03-ef6e-4f49-9a44-68b4797b3b73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:27.529838 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941acb03-ef6e-4f49-9a44-68b4797b3b73 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:27.537092 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2a98af-9bb9-44c6-b868-2fb96ab4213d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:27.538151 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2a98af-9bb9-44c6-b868-2fb96ab4213d admin admin] REQUEST_METHOD: `PUT` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:27.538621 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2a98af-9bb9-44c6-b868-2fb96ab4213d admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:27.539079 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2a98af-9bb9-44c6-b868-2fb96ab4213d admin admin] PATH_INFO: `/v3/projects/bc8eab398ee643eebae35cfb79ca4f7e/users/e33c0fdbf5aa455dafc95427e8a9544d/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:27.574348 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2a98af-9bb9-44c6-b868-2fb96ab4213d admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc8eab398ee643eebae35cfb79ca4f7e, user_id=e33c0fdbf5aa455dafc95427e8a9544d, role_id=5cf1f4bd2fef46ab9c6eff56d52f9f69)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:27.578356 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2a98af-9bb9-44c6-b868-2fb96ab4213d admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:27.585232 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-bc8fb87e-1f57-4be2-a5bd-42f5129f65c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd5ade4db7fe943b2b7d6b33b7c7b5872'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:27.598305 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 763/1521] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 12:00:27 2022] PUT /identity/v3/projects/bc8eab398ee643eebae35cfb79ca4f7e/users/e33c0fdbf5aa455dafc95427e8a9544d/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:00:27.599417 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2a98af-9bb9-44c6-b868-2fb96ab4213d admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:27.615198 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-bc8fb87e-1f57-4be2-a5bd-42f5129f65c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd5ade4db7fe943b2b7d6b33b7c7b5872'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:27.636869 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-bc8fb87e-1f57-4be2-a5bd-42f5129f65c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd5ade4db7fe943b2b7d6b33b7c7b5872'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:27.667078 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-bc8fb87e-1f57-4be2-a5bd-42f5129f65c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd5ade4db7fe943b2b7d6b33b7c7b5872'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:27.669250 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-bc8fb87e-1f57-4be2-a5bd-42f5129f65c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd5ade4db7fe943b2b7d6b33b7c7b5872'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:27.700483 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:00:27.707014 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 759/1522] 10.222.0.59 () {58 vars in 1224 bytes} [Tue Oct 11 12:00:27 2022] DELETE /identity/v3/users/d5ade4db7fe943b2b7d6b33b7c7b5872 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:00:27.722687 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8fb87e-1f57-4be2-a5bd-42f5129f65c6 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:27.725132 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-bc8fb87e-1f57-4be2-a5bd-42f5129f65c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:27.750973 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:00:27.814670 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2a98af-9bb9-44c6-b868-2fb96ab4213d admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:27.980384 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8fb87e-1f57-4be2-a5bd-42f5129f65c6 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:27.997375 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e165cf-a302-45f8-8271-761746411cf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:27.998930 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e165cf-a302-45f8-8271-761746411cf8 admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:27.999657 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e165cf-a302-45f8-8271-761746411cf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:28.000468 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e165cf-a302-45f8-8271-761746411cf8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:28.002335 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e165cf-a302-45f8-8271-761746411cf8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:28.006041 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e165cf-a302-45f8-8271-761746411cf8 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:28.029835 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 760/1523] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:00:27 2022] GET /identity/v3/roles => generated 1420 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:00:28.032268 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e165cf-a302-45f8-8271-761746411cf8 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:28.036193 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-21e165cf-a302-45f8-8271-761746411cf8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:28.064613 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e165cf-a302-45f8-8271-761746411cf8 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:28.089146 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebcd8df-8a40-4771-8267-7759541b90db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:28.091622 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebcd8df-8a40-4771-8267-7759541b90db admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:28.092255 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebcd8df-8a40-4771-8267-7759541b90db admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:28.092837 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebcd8df-8a40-4771-8267-7759541b90db admin admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:28.094133 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebcd8df-8a40-4771-8267-7759541b90db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:28.097243 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebcd8df-8a40-4771-8267-7759541b90db admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:28.174992 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 761/1524] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:00:28 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:00:28.177189 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebcd8df-8a40-4771-8267-7759541b90db admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:28.195045 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9988e8c6-4abc-4dba-b64c-8038f9b2453b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:28.196233 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9988e8c6-4abc-4dba-b64c-8038f9b2453b admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:28.196710 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9988e8c6-4abc-4dba-b64c-8038f9b2453b admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:28.197191 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9988e8c6-4abc-4dba-b64c-8038f9b2453b admin admin] PATH_INFO: `/v3/roles` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:28.198445 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9988e8c6-4abc-4dba-b64c-8038f9b2453b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:28.208772 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9988e8c6-4abc-4dba-b64c-8038f9b2453b admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:28.211787 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebcd8df-8a40-4771-8267-7759541b90db admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:28.229308 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f482c17c-2b62-46ca-930b-44dd4140d856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:28.232005 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f482c17c-2b62-46ca-930b-44dd4140d856 admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:28.232354 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f482c17c-2b62-46ca-930b-44dd4140d856 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:28.232712 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f482c17c-2b62-46ca-930b-44dd4140d856 admin admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:28.233676 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f482c17c-2b62-46ca-930b-44dd4140d856 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:28.235523 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f482c17c-2b62-46ca-930b-44dd4140d856 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:28.254188 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 764/1525] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:00:27 2022] GET /identity/v3/roles => generated 1420 bytes in 655 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:00:28.255835 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9988e8c6-4abc-4dba-b64c-8038f9b2453b admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:28.277496 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 762/1526] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:00:28 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:00:28.281281 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f482c17c-2b62-46ca-930b-44dd4140d856 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:28.287660 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9988e8c6-4abc-4dba-b64c-8038f9b2453b admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:28.289699 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-f482c17c-2b62-46ca-930b-44dd4140d856 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:28.311393 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb34c6e9-628e-4a50-b1ad-8a65891c7b17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:28.312574 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb34c6e9-628e-4a50-b1ad-8a65891c7b17 admin admin] REQUEST_METHOD: `PUT` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:28.313629 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb34c6e9-628e-4a50-b1ad-8a65891c7b17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:28.317286 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb34c6e9-628e-4a50-b1ad-8a65891c7b17 admin admin] PATH_INFO: `/v3/projects/bc8eab398ee643eebae35cfb79ca4f7e/users/e33c0fdbf5aa455dafc95427e8a9544d/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:28.323420 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb34c6e9-628e-4a50-b1ad-8a65891c7b17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc8eab398ee643eebae35cfb79ca4f7e, user_id=e33c0fdbf5aa455dafc95427e8a9544d, role_id=5cf1f4bd2fef46ab9c6eff56d52f9f69)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:28.336041 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f482c17c-2b62-46ca-930b-44dd4140d856 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:28.344845 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb34c6e9-628e-4a50-b1ad-8a65891c7b17 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:28.356398 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663494f3-d10d-4d41-8f91-1341d6e8ceda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:28.358950 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663494f3-d10d-4d41-8f91-1341d6e8ceda admin admin] REQUEST_METHOD: `PUT` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:28.360936 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663494f3-d10d-4d41-8f91-1341d6e8ceda admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:28.361690 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663494f3-d10d-4d41-8f91-1341d6e8ceda admin admin] PATH_INFO: `/v3/projects/4a8ea24a20dc486c9f504488c46fef97/users/9ef6c9ae145e4a3f898d50cef8f3c0e3/roles/de2d2bd26fff4748a79990f97a107164` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:28.366680 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 765/1527] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 12:00:28 2022] PUT /identity/v3/projects/bc8eab398ee643eebae35cfb79ca4f7e/users/e33c0fdbf5aa455dafc95427e8a9544d/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:00:28.366680 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb34c6e9-628e-4a50-b1ad-8a65891c7b17 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:28.416777 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663494f3-d10d-4d41-8f91-1341d6e8ceda admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a8ea24a20dc486c9f504488c46fef97, user_id=9ef6c9ae145e4a3f898d50cef8f3c0e3, role_id=de2d2bd26fff4748a79990f97a107164)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:28.428002 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663494f3-d10d-4d41-8f91-1341d6e8ceda admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:28.504368 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 763/1528] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 12:00:28 2022] PUT /identity/v3/projects/4a8ea24a20dc486c9f504488c46fef97/users/9ef6c9ae145e4a3f898d50cef8f3c0e3/roles/de2d2bd26fff4748a79990f97a107164 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:00:28.510140 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270f5ecc-66d1-4ec4-ae10-61f03c6f055b None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:28.511452 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270f5ecc-66d1-4ec4-ae10-61f03c6f055b None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:28.512050 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270f5ecc-66d1-4ec4-ae10-61f03c6f055b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:28.533388 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb34c6e9-628e-4a50-b1ad-8a65891c7b17 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:28.595087 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-270f5ecc-66d1-4ec4-ae10-61f03c6f055b None None] MFA Rules not processed for user `e33c0fdbf5aa455dafc95427e8a9544d`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:00:28.597737 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-270f5ecc-66d1-4ec4-ae10-61f03c6f055b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:28.700704 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bfc5ee-59b9-4174-a2b2-450f44ef843f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:28.701888 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bfc5ee-59b9-4174-a2b2-450f44ef843f admin admin] REQUEST_METHOD: `PUT` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:28.702366 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bfc5ee-59b9-4174-a2b2-450f44ef843f admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:28.703109 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bfc5ee-59b9-4174-a2b2-450f44ef843f admin admin] PATH_INFO: `/v3/projects/d5a16d2bddcf4457b1bda9239ac43f9a/users/c034ca927819468f8ad97b47ec1cbafd/roles/de2d2bd26fff4748a79990f97a107164` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:28.736656 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:00:28.739946 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bfc5ee-59b9-4174-a2b2-450f44ef843f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5a16d2bddcf4457b1bda9239ac43f9a, user_id=c034ca927819468f8ad97b47ec1cbafd, role_id=de2d2bd26fff4748a79990f97a107164)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:28.748783 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bfc5ee-59b9-4174-a2b2-450f44ef843f admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:28.780319 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 764/1529] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:00:28 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:00:28.785478 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789351fc-0393-4aef-9ef0-e560c9addaad None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:28.786088 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789351fc-0393-4aef-9ef0-e560c9addaad None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:28.786575 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789351fc-0393-4aef-9ef0-e560c9addaad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:28.804060 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 766/1530] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 12:00:28 2022] PUT /identity/v3/projects/d5a16d2bddcf4457b1bda9239ac43f9a/users/c034ca927819468f8ad97b47ec1cbafd/roles/de2d2bd26fff4748a79990f97a107164 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:00:28.816288 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bfc5ee-59b9-4174-a2b2-450f44ef843f admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:28.821119 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-789351fc-0393-4aef-9ef0-e560c9addaad None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:00:28.822851 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-789351fc-0393-4aef-9ef0-e560c9addaad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:28.827041 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-31bfc5ee-59b9-4174-a2b2-450f44ef843f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:28.906200 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:00:28.953286 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bfc5ee-59b9-4174-a2b2-450f44ef843f admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:28.973868 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4527a0-0057-4467-b324-1ea447174b10 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b1b651c73664d08accc6edf2f79bace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:28.975013 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4527a0-0057-4467-b324-1ea447174b10 service cinder] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:28.975204 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4527a0-0057-4467-b324-1ea447174b10 service cinder] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:28.975824 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4527a0-0057-4467-b324-1ea447174b10 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:28.981104 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 765/1531] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:00:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:00:28.994827 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e392093-484f-4f74-901f-e95e742f0b30 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:28.996268 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e392093-484f-4f74-901f-e95e742f0b30 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:28.999636 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e392093-484f-4f74-901f-e95e742f0b30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:29.094038 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-5e392093-484f-4f74-901f-e95e742f0b30 None None] MFA Rules not processed for user `c034ca927819468f8ad97b47ec1cbafd`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:00:29.096082 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-5e392093-484f-4f74-901f-e95e742f0b30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:29.294603 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 766/1532] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:00:28 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:00:29.294603 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e392093-484f-4f74-901f-e95e742f0b30 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:29.308493 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-5e392093-484f-4f74-901f-e95e742f0b30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:29.311584 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4527a0-0057-4467-b324-1ea447174b10 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:29.317576 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4527a0-0057-4467-b324-1ea447174b10 service cinder] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:29.339293 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 767/1533] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:00:28 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 534 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:00:29.341031 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4527a0-0057-4467-b324-1ea447174b10 service cinder] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:29.343873 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-1b4527a0-0057-4467-b324-1ea447174b10 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:29.368462 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e392093-484f-4f74-901f-e95e742f0b30 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:29.383047 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133c2b00-fcb6-46d6-9ded-d0c370a8da66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:29.384048 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133c2b00-fcb6-46d6-9ded-d0c370a8da66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:29.384652 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133c2b00-fcb6-46d6-9ded-d0c370a8da66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:29.391378 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133c2b00-fcb6-46d6-9ded-d0c370a8da66 admin admin] PATH_INFO: `/v3/projects/54ad85255745456a98daf8c94b8a1c25` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:29.396385 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133c2b00-fcb6-46d6-9ded-d0c370a8da66 admin admin] RBAC: Authorizing `identity:delete_project(project_id=54ad85255745456a98daf8c94b8a1c25)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:29.403539 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133c2b00-fcb6-46d6-9ded-d0c370a8da66 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:29.530195 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-133c2b00-fcb6-46d6-9ded-d0c370a8da66 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '54ad85255745456a98daf8c94b8a1c25', 'request_id': 'req-133c2b00-fcb6-46d6-9ded-d0c370a8da66'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:29.543612 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-133c2b00-fcb6-46d6-9ded-d0c370a8da66 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '54ad85255745456a98daf8c94b8a1c25', 'request_id': 'req-133c2b00-fcb6-46d6-9ded-d0c370a8da66'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:29.568817 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-133c2b00-fcb6-46d6-9ded-d0c370a8da66 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '54ad85255745456a98daf8c94b8a1c25', 'request_id': 'req-133c2b00-fcb6-46d6-9ded-d0c370a8da66'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:29.623104 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4527a0-0057-4467-b324-1ea447174b10 service cinder] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:29.623703 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-133c2b00-fcb6-46d6-9ded-d0c370a8da66 admin admin] The token cache is being invalidate because project 54ad85255745456a98daf8c94b8a1c25 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81432) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 12:00:29.623703 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-133c2b00-fcb6-46d6-9ded-d0c370a8da66 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:29.624827 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-133c2b00-fcb6-46d6-9ded-d0c370a8da66 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:29.631994 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 767/1534] 10.222.0.59 () {58 vars in 1233 bytes} [Tue Oct 11 12:00:29 2022] DELETE /identity/v3/projects/54ad85255745456a98daf8c94b8a1c25 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:00:29.634438 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60fdd59-9091-47d5-8269-646cfb3a9145 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:29.634958 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60fdd59-9091-47d5-8269-646cfb3a9145 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:29.635453 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60fdd59-9091-47d5-8269-646cfb3a9145 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:29.715194 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-b60fdd59-9091-47d5-8269-646cfb3a9145 None None] MFA Rules not processed for user `2e563fe174db4cc19ae6bf8162cc3d23`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:00:29.717442 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-b60fdd59-9091-47d5-8269-646cfb3a9145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:29.861941 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc21c8cd-4537-4911-949a-dc14f978b4a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e563fe174db4cc19ae6bf8162cc3d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:29.864018 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc21c8cd-4537-4911-949a-dc14f978b4a4 service neutron] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:29.864940 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc21c8cd-4537-4911-949a-dc14f978b4a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:29.867523 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc21c8cd-4537-4911-949a-dc14f978b4a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:29.880054 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-cc21c8cd-4537-4911-949a-dc14f978b4a4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:30.007342 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:00:30.030869 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:00:30.065853 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 768/1535] 10.222.0.50 () {58 vars in 1015 bytes} [Tue Oct 11 12:00:29 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 433 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:00:30.067394 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60fdd59-9091-47d5-8269-646cfb3a9145 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:30.073056 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-b60fdd59-9091-47d5-8269-646cfb3a9145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:30.180862 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc21c8cd-4537-4911-949a-dc14f978b4a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:30.184032 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc21c8cd-4537-4911-949a-dc14f978b4a4 service neutron] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:30.204426 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 768/1536] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:00:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 864 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:00:30.206998 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53df876-fd93-4229-b14f-c689c50aa216 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:30.207542 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53df876-fd93-4229-b14f-c689c50aa216 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:30.208057 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53df876-fd93-4229-b14f-c689c50aa216 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:30.283913 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60fdd59-9091-47d5-8269-646cfb3a9145 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:30.295186 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799e4403-db19-47af-bf71-40006ea1c9fa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b1b651c73664d08accc6edf2f79bace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:30.296064 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799e4403-db19-47af-bf71-40006ea1c9fa service cinder] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:30.296474 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799e4403-db19-47af-bf71-40006ea1c9fa service cinder] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:30.296855 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799e4403-db19-47af-bf71-40006ea1c9fa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:30.306129 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-a53df876-fd93-4229-b14f-c689c50aa216 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:00:30.309368 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-799e4403-db19-47af-bf71-40006ea1c9fa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:30.310886 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-a53df876-fd93-4229-b14f-c689c50aa216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:30.348662 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 769/1537] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:00:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:00:30.350341 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53df876-fd93-4229-b14f-c689c50aa216 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:30.357208 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-a53df876-fd93-4229-b14f-c689c50aa216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:30.421061 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53df876-fd93-4229-b14f-c689c50aa216 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:30.446059 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae54262-95f0-4f6d-9814-44b94467f73d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b1b651c73664d08accc6edf2f79bace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:30.447328 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae54262-95f0-4f6d-9814-44b94467f73d service cinder] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:30.447769 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae54262-95f0-4f6d-9814-44b94467f73d service cinder] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:30.449319 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae54262-95f0-4f6d-9814-44b94467f73d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:30.458477 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-9ae54262-95f0-4f6d-9814-44b94467f73d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:30.485372 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799e4403-db19-47af-bf71-40006ea1c9fa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:30.492998 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799e4403-db19-47af-bf71-40006ea1c9fa service cinder] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:30.517214 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 769/1538] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:00:30 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:00:30.518863 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ca8af5-1011-4c34-8c80-60cd15e99bf0 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:30.519249 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ca8af5-1011-4c34-8c80-60cd15e99bf0 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:30.519440 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ca8af5-1011-4c34-8c80-60cd15e99bf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:30.574964 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-21ca8af5-1011-4c34-8c80-60cd15e99bf0 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:00:30.580343 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-21ca8af5-1011-4c34-8c80-60cd15e99bf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:30.609153 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 770/1539] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:00:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:00:30.610472 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ca8af5-1011-4c34-8c80-60cd15e99bf0 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:30.612763 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-21ca8af5-1011-4c34-8c80-60cd15e99bf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:30.634781 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ca8af5-1011-4c34-8c80-60cd15e99bf0 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:30.638400 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae54262-95f0-4f6d-9814-44b94467f73d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:30.640644 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf1689c-6297-4191-8aec-a2978f6113a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e563fe174db4cc19ae6bf8162cc3d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:30.641386 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf1689c-6297-4191-8aec-a2978f6113a7 service neutron] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:30.641782 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf1689c-6297-4191-8aec-a2978f6113a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:30.642030 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf1689c-6297-4191-8aec-a2978f6113a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:30.646111 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae54262-95f0-4f6d-9814-44b94467f73d service cinder] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:30.652095 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf1689c-6297-4191-8aec-a2978f6113a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:30.654895 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf1689c-6297-4191-8aec-a2978f6113a7 service neutron] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:30.674293 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 771/1540] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:00:30 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:00:30.675456 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf1689c-6297-4191-8aec-a2978f6113a7 service neutron] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:30.690384 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf1689c-6297-4191-8aec-a2978f6113a7 service neutron] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:30.698374 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1dbad4-712f-42dc-8cdd-3296bcc527e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e563fe174db4cc19ae6bf8162cc3d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:30.700888 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1dbad4-712f-42dc-8cdd-3296bcc527e6 service neutron] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:30.701567 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1dbad4-712f-42dc-8cdd-3296bcc527e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:30.702012 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1dbad4-712f-42dc-8cdd-3296bcc527e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:30.708516 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 770/1541] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:00:30 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:00:30.710125 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae54262-95f0-4f6d-9814-44b94467f73d service cinder] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:30.733220 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-9ae54262-95f0-4f6d-9814-44b94467f73d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:30.736983 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1dbad4-712f-42dc-8cdd-3296bcc527e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:30.739140 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1dbad4-712f-42dc-8cdd-3296bcc527e6 service neutron] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:30.800936 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 772/1542] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:00:30 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:00:30.803590 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1dbad4-712f-42dc-8cdd-3296bcc527e6 service neutron] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:30.808370 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae54262-95f0-4f6d-9814-44b94467f73d service cinder] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:30.831968 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae48be3-fc5e-4291-b62e-e851779380ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e563fe174db4cc19ae6bf8162cc3d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:30.832566 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae48be3-fc5e-4291-b62e-e851779380ab service neutron] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:30.832964 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae48be3-fc5e-4291-b62e-e851779380ab service neutron] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:30.833263 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae48be3-fc5e-4291-b62e-e851779380ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:30.870798 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1dbad4-712f-42dc-8cdd-3296bcc527e6 service neutron] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:30.876218 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae48be3-fc5e-4291-b62e-e851779380ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:30.881490 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae48be3-fc5e-4291-b62e-e851779380ab service neutron] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:30.890301 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cca5e9-3456-4e3d-9bfa-23adee8a199c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:30.891051 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cca5e9-3456-4e3d-9bfa-23adee8a199c admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:30.891388 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cca5e9-3456-4e3d-9bfa-23adee8a199c admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:30.894767 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cca5e9-3456-4e3d-9bfa-23adee8a199c admin admin] PATH_INFO: `/v3/domains` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:30.895754 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cca5e9-3456-4e3d-9bfa-23adee8a199c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 12:00:30.896105 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cca5e9-3456-4e3d-9bfa-23adee8a199c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:30.903414 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cca5e9-3456-4e3d-9bfa-23adee8a199c admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:30.918885 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 771/1543] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:00:30 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:00:30.959565 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 773/1544] 10.222.0.59 () {58 vars in 1153 bytes} [Tue Oct 11 12:00:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 12:00:30.960863 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae48be3-fc5e-4291-b62e-e851779380ab service neutron] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:30.985003 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae48be3-fc5e-4291-b62e-e851779380ab service neutron] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:31.019189 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9e7380-af0e-4f69-b542-3a16a12f9611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:31.020469 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9e7380-af0e-4f69-b542-3a16a12f9611 admin admin] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:31.021127 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9e7380-af0e-4f69-b542-3a16a12f9611 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:31.022341 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9e7380-af0e-4f69-b542-3a16a12f9611 admin admin] PATH_INFO: `/v3/projects` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:31.023246 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9e7380-af0e-4f69-b542-3a16a12f9611 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:31.033051 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9e7380-af0e-4f69-b542-3a16a12f9611 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:31.156637 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 772/1545] 10.222.0.59 () {60 vars in 1153 bytes} [Tue Oct 11 12:00:30 2022] POST /identity/v3/projects => generated 401 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:00:31.162449 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cca5e9-3456-4e3d-9bfa-23adee8a199c admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:31.190577 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cca5e9-3456-4e3d-9bfa-23adee8a199c admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:31.466306 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2df30c-1a24-4b1d-9ec2-eb49e7cc15e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:31.467155 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2df30c-1a24-4b1d-9ec2-eb49e7cc15e4 admin admin] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:31.467511 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2df30c-1a24-4b1d-9ec2-eb49e7cc15e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:31.467842 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2df30c-1a24-4b1d-9ec2-eb49e7cc15e4 admin admin] PATH_INFO: `/v3/users` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:31.468517 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2df30c-1a24-4b1d-9ec2-eb49e7cc15e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:31.472193 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2df30c-1a24-4b1d-9ec2-eb49e7cc15e4 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:31.541098 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:00:31.562522 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 774/1546] 10.222.0.59 () {60 vars in 1144 bytes} [Tue Oct 11 12:00:31 2022] POST /identity/v3/users => generated 301 bytes in 400 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:00:31.564038 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9e7380-af0e-4f69-b542-3a16a12f9611 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:31.616249 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9e7380-af0e-4f69-b542-3a16a12f9611 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:31.640912 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef2d630-b7ee-4b34-878f-72e36fbc40fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:31.646426 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef2d630-b7ee-4b34-878f-72e36fbc40fe admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:31.649194 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef2d630-b7ee-4b34-878f-72e36fbc40fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:31.649598 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef2d630-b7ee-4b34-878f-72e36fbc40fe admin admin] PATH_INFO: `/v3/roles` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:31.650398 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef2d630-b7ee-4b34-878f-72e36fbc40fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:31.652175 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef2d630-b7ee-4b34-878f-72e36fbc40fe admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:31.707209 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 773/1547] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:00:31 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:00:31.712085 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2df30c-1a24-4b1d-9ec2-eb49e7cc15e4 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:31.743487 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2df30c-1a24-4b1d-9ec2-eb49e7cc15e4 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:31.774882 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de478b1f-3038-4850-92dc-dc3950fd3377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:31.775930 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de478b1f-3038-4850-92dc-dc3950fd3377 admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:31.776398 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de478b1f-3038-4850-92dc-dc3950fd3377 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:31.776748 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de478b1f-3038-4850-92dc-dc3950fd3377 admin admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:31.777535 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de478b1f-3038-4850-92dc-dc3950fd3377 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:31.779759 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de478b1f-3038-4850-92dc-dc3950fd3377 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:31.828993 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 775/1548] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:00:31 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:00:31.834344 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef2d630-b7ee-4b34-878f-72e36fbc40fe admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:31.868533 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef2d630-b7ee-4b34-878f-72e36fbc40fe admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:31.889187 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad3e5e0-8ad9-406a-a1e2-d6328e1beae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:31.898601 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad3e5e0-8ad9-406a-a1e2-d6328e1beae2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:31.898601 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad3e5e0-8ad9-406a-a1e2-d6328e1beae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:31.898601 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad3e5e0-8ad9-406a-a1e2-d6328e1beae2 admin admin] PATH_INFO: `/v3/projects/5a73174d8ceb48748d21449cd773364e/users/da5d687afab048a2addda468a586661e/roles/de2d2bd26fff4748a79990f97a107164` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:31.973985 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad3e5e0-8ad9-406a-a1e2-d6328e1beae2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a73174d8ceb48748d21449cd773364e, user_id=da5d687afab048a2addda468a586661e, role_id=de2d2bd26fff4748a79990f97a107164)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:31.985594 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad3e5e0-8ad9-406a-a1e2-d6328e1beae2 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:32.034591 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 774/1549] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 12:00:31 2022] PUT /identity/v3/projects/5a73174d8ceb48748d21449cd773364e/users/da5d687afab048a2addda468a586661e/roles/de2d2bd26fff4748a79990f97a107164 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 12:00:32.041443 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a540715-703e-4bde-a41e-a4e450e705de None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:32.042121 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a540715-703e-4bde-a41e-a4e450e705de None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:32.042746 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a540715-703e-4bde-a41e-a4e450e705de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:32.115857 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-2a540715-703e-4bde-a41e-a4e450e705de None None] MFA Rules not processed for user `da5d687afab048a2addda468a586661e`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:00:32.120367 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-2a540715-703e-4bde-a41e-a4e450e705de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:32.421998 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 776/1550] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:00:32 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 382 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 12:00:32.427992 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad3e5e0-8ad9-406a-a1e2-d6328e1beae2 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:32.610100 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:00:32.772466 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad3e5e0-8ad9-406a-a1e2-d6328e1beae2 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:32.822116 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e22265-35ea-46e2-a7ba-6f9b1fc66aed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:32.823422 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e22265-35ea-46e2-a7ba-6f9b1fc66aed admin admin] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:32.823947 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e22265-35ea-46e2-a7ba-6f9b1fc66aed admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:32.829209 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e22265-35ea-46e2-a7ba-6f9b1fc66aed admin admin] PATH_INFO: `/v3/projects` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:32.839752 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e22265-35ea-46e2-a7ba-6f9b1fc66aed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:32.853745 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e22265-35ea-46e2-a7ba-6f9b1fc66aed admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:33.039277 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 775/1551] 10.222.0.59 () {60 vars in 1153 bytes} [Tue Oct 11 12:00:32 2022] POST /identity/v3/projects => generated 401 bytes in 608 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:00:33.042999 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a540715-703e-4bde-a41e-a4e450e705de None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:33.136355 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a540715-703e-4bde-a41e-a4e450e705de None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:33.528620 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc43494-2e14-4bd9-a9ce-7cb5799dae9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:33.534592 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc43494-2e14-4bd9-a9ce-7cb5799dae9a admin admin] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:33.539668 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc43494-2e14-4bd9-a9ce-7cb5799dae9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:33.540308 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc43494-2e14-4bd9-a9ce-7cb5799dae9a admin admin] PATH_INFO: `/v3/users` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:33.545663 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc43494-2e14-4bd9-a9ce-7cb5799dae9a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:33.554162 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc43494-2e14-4bd9-a9ce-7cb5799dae9a admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:33.645988 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 777/1552] 10.222.0.59 () {60 vars in 1144 bytes} [Tue Oct 11 12:00:33 2022] POST /identity/v3/users => generated 307 bytes in 606 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:00:33.646284 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e22265-35ea-46e2-a7ba-6f9b1fc66aed admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:33.703659 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e22265-35ea-46e2-a7ba-6f9b1fc66aed admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:33.711221 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bcb2b0-511d-4574-8e09-e0d5310a9e45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:33.712246 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bcb2b0-511d-4574-8e09-e0d5310a9e45 admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:33.712654 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bcb2b0-511d-4574-8e09-e0d5310a9e45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:33.713184 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bcb2b0-511d-4574-8e09-e0d5310a9e45 admin admin] PATH_INFO: `/v3/roles` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:33.714893 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bcb2b0-511d-4574-8e09-e0d5310a9e45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:33.716692 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bcb2b0-511d-4574-8e09-e0d5310a9e45 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:33.768793 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 776/1553] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:00:33 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:00:33.775590 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc43494-2e14-4bd9-a9ce-7cb5799dae9a admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:33.789612 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc43494-2e14-4bd9-a9ce-7cb5799dae9a admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:33.807162 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcfc86c-1965-4b31-8a7a-25344bd1063b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:33.807966 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcfc86c-1965-4b31-8a7a-25344bd1063b admin admin] REQUEST_METHOD: `PUT` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:33.808479 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcfc86c-1965-4b31-8a7a-25344bd1063b admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:33.808834 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcfc86c-1965-4b31-8a7a-25344bd1063b admin admin] PATH_INFO: `/v3/projects/54a3e98b3aaa421c9f2e3ef580921ac6/users/e6f1f89743a540e4acac7bdd0c6c139d/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:33.853045 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcfc86c-1965-4b31-8a7a-25344bd1063b admin admin] RBAC: Authorizing `identity:create_grant(project_id=54a3e98b3aaa421c9f2e3ef580921ac6, user_id=e6f1f89743a540e4acac7bdd0c6c139d, role_id=5cf1f4bd2fef46ab9c6eff56d52f9f69)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:33.856431 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcfc86c-1965-4b31-8a7a-25344bd1063b admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:33.901655 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 778/1554] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 12:00:33 2022] PUT /identity/v3/projects/54a3e98b3aaa421c9f2e3ef580921ac6/users/e6f1f89743a540e4acac7bdd0c6c139d/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 12:00:33.907822 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bcb2b0-511d-4574-8e09-e0d5310a9e45 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:33.945073 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bcb2b0-511d-4574-8e09-e0d5310a9e45 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:34.242804 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c055d0-fdcd-45db-972c-50a09d498308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:34.243474 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c055d0-fdcd-45db-972c-50a09d498308 admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:34.243764 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c055d0-fdcd-45db-972c-50a09d498308 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:34.244044 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c055d0-fdcd-45db-972c-50a09d498308 admin admin] PATH_INFO: `/v3/roles` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:34.245001 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c055d0-fdcd-45db-972c-50a09d498308 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:34.248510 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c055d0-fdcd-45db-972c-50a09d498308 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:34.282409 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 777/1555] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:00:33 2022] GET /identity/v3/roles => generated 1420 bytes in 370 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:00:34.284250 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcfc86c-1965-4b31-8a7a-25344bd1063b admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:34.332293 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcfc86c-1965-4b31-8a7a-25344bd1063b admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:34.353352 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b88ed04-df60-483f-97d6-598d331009f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:34.355042 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b88ed04-df60-483f-97d6-598d331009f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:34.359865 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b88ed04-df60-483f-97d6-598d331009f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:34.360676 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b88ed04-df60-483f-97d6-598d331009f9 admin admin] PATH_INFO: `/v3/projects/54a3e98b3aaa421c9f2e3ef580921ac6/users/e6f1f89743a540e4acac7bdd0c6c139d/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:34.369271 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b88ed04-df60-483f-97d6-598d331009f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=54a3e98b3aaa421c9f2e3ef580921ac6, user_id=e6f1f89743a540e4acac7bdd0c6c139d, role_id=5cf1f4bd2fef46ab9c6eff56d52f9f69)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:34.382606 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b88ed04-df60-483f-97d6-598d331009f9 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:34.429498 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 779/1556] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 12:00:34 2022] PUT /identity/v3/projects/54a3e98b3aaa421c9f2e3ef580921ac6/users/e6f1f89743a540e4acac7bdd0c6c139d/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 12:00:34.455948 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e13e9b-5f11-46f3-812b-1b257bac16df None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:34.456637 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e13e9b-5f11-46f3-812b-1b257bac16df None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:34.457737 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e13e9b-5f11-46f3-812b-1b257bac16df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:34.554432 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-b1e13e9b-5f11-46f3-812b-1b257bac16df None None] MFA Rules not processed for user `e6f1f89743a540e4acac7bdd0c6c139d`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:00:34.556610 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-b1e13e9b-5f11-46f3-812b-1b257bac16df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:34.724243 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 778/1557] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:00:34 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:00:34.741187 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b88ed04-df60-483f-97d6-598d331009f9 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:34.832360 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:00:34.931356 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b88ed04-df60-483f-97d6-598d331009f9 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:34.943392 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f542b955-195e-48c3-9ec5-5a9a6461f254 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b1b651c73664d08accc6edf2f79bace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:34.944980 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f542b955-195e-48c3-9ec5-5a9a6461f254 service cinder] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:34.945812 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f542b955-195e-48c3-9ec5-5a9a6461f254 service cinder] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:34.946600 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f542b955-195e-48c3-9ec5-5a9a6461f254 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:34.977960 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f542b955-195e-48c3-9ec5-5a9a6461f254 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:34.980868 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f542b955-195e-48c3-9ec5-5a9a6461f254 service cinder] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:35.005028 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 780/1558] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:00:34 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:00:35.238109 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e13e9b-5f11-46f3-812b-1b257bac16df None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:35.274599 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e13e9b-5f11-46f3-812b-1b257bac16df None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:35.286154 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c7cbbc-679b-4c9f-860e-b86c8d99a5e9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b1b651c73664d08accc6edf2f79bace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:35.289494 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c7cbbc-679b-4c9f-860e-b86c8d99a5e9 service cinder] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:35.290618 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c7cbbc-679b-4c9f-860e-b86c8d99a5e9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:35.292765 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c7cbbc-679b-4c9f-860e-b86c8d99a5e9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:35.407351 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:00:35.531294 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c7cbbc-679b-4c9f-860e-b86c8d99a5e9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:35.540561 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c7cbbc-679b-4c9f-860e-b86c8d99a5e9 service cinder] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:35.555014 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 779/1559] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:00:35 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:00:35.556710 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4eabe0-feaa-4d9d-9dd3-0caf9ad2fa6a None None] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:35.557196 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4eabe0-feaa-4d9d-9dd3-0caf9ad2fa6a None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:35.557685 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4eabe0-feaa-4d9d-9dd3-0caf9ad2fa6a None None] PATH_INFO: `/` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:35.560643 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 781/1560] 10.222.0.59 () {54 vars in 921 bytes} [Tue Oct 11 12:00:35 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 11 12:00:35.625483 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f45068-de98-4f11-bd97-7bb17217d765 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:35.625869 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f45068-de98-4f11-bd97-7bb17217d765 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:35.626539 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f45068-de98-4f11-bd97-7bb17217d765 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:35.702028 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01050bd-2d33-4fa7-882e-d71daf9a3b43 None None] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:35.703214 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01050bd-2d33-4fa7-882e-d71daf9a3b43 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:35.703965 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01050bd-2d33-4fa7-882e-d71daf9a3b43 None None] PATH_INFO: `/` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:35.708842 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 782/1561] 10.222.0.59 () {54 vars in 921 bytes} [Tue Oct 11 12:00:35 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 12:00:35.709074 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-e3f45068-de98-4f11-bd97-7bb17217d765 None None] MFA Rules not processed for user `bad5e1cb306b46818096940acc3810b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:00:35.710702 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-e3f45068-de98-4f11-bd97-7bb17217d765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:35.776084 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0baa41b-06bf-4c6e-a33f-43f3265f19e8 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:35.777447 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0baa41b-06bf-4c6e-a33f-43f3265f19e8 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:35.778510 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0baa41b-06bf-4c6e-a33f-43f3265f19e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:35.865880 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-a0baa41b-06bf-4c6e-a33f-43f3265f19e8 None None] MFA Rules not processed for user `bad5e1cb306b46818096940acc3810b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:00:35.869283 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-a0baa41b-06bf-4c6e-a33f-43f3265f19e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:35.935773 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 780/1562] 10.222.0.59 () {58 vars in 1018 bytes} [Tue Oct 11 12:00:35 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 12:00:35.980900 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f45068-de98-4f11-bd97-7bb17217d765 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:35.983349 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-e3f45068-de98-4f11-bd97-7bb17217d765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:36.052701 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f45068-de98-4f11-bd97-7bb17217d765 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:36.076080 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c20ef0-acbb-426f-9e66-46c8bdcffee2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bad5e1cb306b46818096940acc3810b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:36.076791 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c20ef0-acbb-426f-9e66-46c8bdcffee2 service nova] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:36.078828 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c20ef0-acbb-426f-9e66-46c8bdcffee2 service nova] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:36.079324 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c20ef0-acbb-426f-9e66-46c8bdcffee2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:36.099412 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c20ef0-acbb-426f-9e66-46c8bdcffee2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:36.101825 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c20ef0-acbb-426f-9e66-46c8bdcffee2 service nova] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:36.130027 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 783/1563] 10.222.0.59 () {58 vars in 1018 bytes} [Tue Oct 11 12:00:35 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 356 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 12:00:36.132573 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e07a7f-f3e5-463b-a486-635c97e96031 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:36.132939 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e07a7f-f3e5-463b-a486-635c97e96031 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:36.134542 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e07a7f-f3e5-463b-a486-635c97e96031 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:36.148714 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 781/1564] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:00:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 11 12:00:36.226075 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-d1e07a7f-f3e5-463b-a486-635c97e96031 None None] MFA Rules not processed for user `9ef6c9ae145e4a3f898d50cef8f3c0e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:00:36.228050 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-d1e07a7f-f3e5-463b-a486-635c97e96031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:36.358544 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c20ef0-acbb-426f-9e66-46c8bdcffee2 service nova] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:36.382964 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c20ef0-acbb-426f-9e66-46c8bdcffee2 service nova] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:36.397778 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44276980-67ea-4eb7-a9e0-1870f6293449 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bad5e1cb306b46818096940acc3810b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:36.399380 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44276980-67ea-4eb7-a9e0-1870f6293449 service nova] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:36.402213 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44276980-67ea-4eb7-a9e0-1870f6293449 service nova] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:36.403339 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44276980-67ea-4eb7-a9e0-1870f6293449 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:36.482198 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44276980-67ea-4eb7-a9e0-1870f6293449 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:36.485537 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44276980-67ea-4eb7-a9e0-1870f6293449 service nova] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:36.576926 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 784/1565] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:00:36 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 446 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:00:36.583220 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e07a7f-f3e5-463b-a486-635c97e96031 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:36.585374 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 782/1566] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:00:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 11 12:00:36.590272 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-d1e07a7f-f3e5-463b-a486-635c97e96031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:36.783068 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:00:36.814091 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e07a7f-f3e5-463b-a486-635c97e96031 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:36.843959 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276b0382-71fd-4ee6-977d-866c3ab62449 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:36.844973 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276b0382-71fd-4ee6-977d-866c3ab62449 admin admin] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:36.848271 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276b0382-71fd-4ee6-977d-866c3ab62449 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:36.848708 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276b0382-71fd-4ee6-977d-866c3ab62449 admin admin] PATH_INFO: `/v3/projects` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:36.849783 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276b0382-71fd-4ee6-977d-866c3ab62449 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:36.861601 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276b0382-71fd-4ee6-977d-866c3ab62449 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:36.934165 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 785/1567] 10.222.0.59 () {60 vars in 1153 bytes} [Tue Oct 11 12:00:36 2022] POST /identity/v3/projects => generated 373 bytes in 352 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:00:36.938751 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44276980-67ea-4eb7-a9e0-1870f6293449 service nova] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:36.995173 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44276980-67ea-4eb7-a9e0-1870f6293449 service nova] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:37.131007 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7703ee-a82b-4d45-a46e-96220b29f645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:37.131007 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7703ee-a82b-4d45-a46e-96220b29f645 admin admin] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:37.131007 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7703ee-a82b-4d45-a46e-96220b29f645 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:37.131007 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7703ee-a82b-4d45-a46e-96220b29f645 admin admin] PATH_INFO: `/v3/users` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:37.131581 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7703ee-a82b-4d45-a46e-96220b29f645 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:37.131581 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7703ee-a82b-4d45-a46e-96220b29f645 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:37.155300 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 783/1568] 10.222.0.59 () {60 vars in 1144 bytes} [Tue Oct 11 12:00:36 2022] POST /identity/v3/users => generated 293 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:00:37.160531 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276b0382-71fd-4ee6-977d-866c3ab62449 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:37.175429 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276b0382-71fd-4ee6-977d-866c3ab62449 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:37.181834 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0894b078-b3e1-462c-9915-e7a3f595f614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:37.182607 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0894b078-b3e1-462c-9915-e7a3f595f614 admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:37.183108 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0894b078-b3e1-462c-9915-e7a3f595f614 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:37.183564 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0894b078-b3e1-462c-9915-e7a3f595f614 admin admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:37.184647 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0894b078-b3e1-462c-9915-e7a3f595f614 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:37.187617 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0894b078-b3e1-462c-9915-e7a3f595f614 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:37.210545 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 786/1569] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:00:37 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:00:37.212175 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7703ee-a82b-4d45-a46e-96220b29f645 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:37.226772 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7703ee-a82b-4d45-a46e-96220b29f645 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:37.232335 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339b9ec7-984a-405a-9f59-1887b0337371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:37.233090 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339b9ec7-984a-405a-9f59-1887b0337371 admin admin] REQUEST_METHOD: `PUT` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:37.233406 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339b9ec7-984a-405a-9f59-1887b0337371 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:37.233721 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339b9ec7-984a-405a-9f59-1887b0337371 admin admin] PATH_INFO: `/v3/projects/c7834515e94848c38d4b619514eb7cfc/users/b0be132a1e3c4ba882c61b1e19de35ac/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:37.251363 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339b9ec7-984a-405a-9f59-1887b0337371 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7834515e94848c38d4b619514eb7cfc, user_id=b0be132a1e3c4ba882c61b1e19de35ac, role_id=5cf1f4bd2fef46ab9c6eff56d52f9f69)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:37.254938 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339b9ec7-984a-405a-9f59-1887b0337371 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:37.257642 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:00:37.277140 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 784/1570] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 12:00:37 2022] PUT /identity/v3/projects/c7834515e94848c38d4b619514eb7cfc/users/b0be132a1e3c4ba882c61b1e19de35ac/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:00:37.282219 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0894b078-b3e1-462c-9915-e7a3f595f614 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:37.294214 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0894b078-b3e1-462c-9915-e7a3f595f614 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:37.364216 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc3cafc-8634-48e3-adb3-688846fcecce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:37.364953 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc3cafc-8634-48e3-adb3-688846fcecce admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:37.365272 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc3cafc-8634-48e3-adb3-688846fcecce admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:37.365730 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc3cafc-8634-48e3-adb3-688846fcecce admin admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:37.366701 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc3cafc-8634-48e3-adb3-688846fcecce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:37.368596 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc3cafc-8634-48e3-adb3-688846fcecce admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:37.380599 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 787/1571] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:00:37 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 11 12:00:37.385631 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339b9ec7-984a-405a-9f59-1887b0337371 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:37.397764 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339b9ec7-984a-405a-9f59-1887b0337371 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:37.403318 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608d11ab-e4e8-466c-bdbb-490070a2bfc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:37.404051 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608d11ab-e4e8-466c-bdbb-490070a2bfc0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:37.404328 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608d11ab-e4e8-466c-bdbb-490070a2bfc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:37.404601 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608d11ab-e4e8-466c-bdbb-490070a2bfc0 admin admin] PATH_INFO: `/v3/projects/c7834515e94848c38d4b619514eb7cfc/users/b0be132a1e3c4ba882c61b1e19de35ac/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:37.406740 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608d11ab-e4e8-466c-bdbb-490070a2bfc0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7834515e94848c38d4b619514eb7cfc, user_id=b0be132a1e3c4ba882c61b1e19de35ac, role_id=5cf1f4bd2fef46ab9c6eff56d52f9f69)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:37.409801 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608d11ab-e4e8-466c-bdbb-490070a2bfc0 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:37.423836 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 785/1572] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 12:00:37 2022] PUT /identity/v3/projects/c7834515e94848c38d4b619514eb7cfc/users/b0be132a1e3c4ba882c61b1e19de35ac/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:00:38.474001 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef0248c-80f0-47d4-90bb-a5177d56c310 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:38.474687 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef0248c-80f0-47d4-90bb-a5177d56c310 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:38.479492 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef0248c-80f0-47d4-90bb-a5177d56c310 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:38.531268 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-3ef0248c-80f0-47d4-90bb-a5177d56c310 None None] MFA Rules not processed for user `b0be132a1e3c4ba882c61b1e19de35ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:00:38.533376 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-3ef0248c-80f0-47d4-90bb-a5177d56c310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:38.633004 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 788/1573] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:00:38 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:00:38.652419 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608d11ab-e4e8-466c-bdbb-490070a2bfc0 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:38.755333 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608d11ab-e4e8-466c-bdbb-490070a2bfc0 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:38.759911 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9420a36d-165d-4249-93cb-cb5625a8220a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b1b651c73664d08accc6edf2f79bace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:38.760578 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9420a36d-165d-4249-93cb-cb5625a8220a service cinder] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:38.760865 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9420a36d-165d-4249-93cb-cb5625a8220a service cinder] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:38.761154 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9420a36d-165d-4249-93cb-cb5625a8220a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:38.974887 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9420a36d-165d-4249-93cb-cb5625a8220a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:38.983164 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9420a36d-165d-4249-93cb-cb5625a8220a service cinder] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:39.016529 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 786/1574] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:00:38 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:00:43.153461 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef0248c-80f0-47d4-90bb-a5177d56c310 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:43.167999 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef0248c-80f0-47d4-90bb-a5177d56c310 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:43.177898 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922586e9-aabe-4550-9a07-fa4fb4af2f94 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b1b651c73664d08accc6edf2f79bace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:43.178862 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922586e9-aabe-4550-9a07-fa4fb4af2f94 service cinder] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:43.179355 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922586e9-aabe-4550-9a07-fa4fb4af2f94 service cinder] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:43.179701 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922586e9-aabe-4550-9a07-fa4fb4af2f94 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:43.195410 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922586e9-aabe-4550-9a07-fa4fb4af2f94 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:43.197462 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922586e9-aabe-4550-9a07-fa4fb4af2f94 service cinder] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:43.211310 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 789/1575] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:00:43 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:00:49.006752 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:INSERT=2,keystone:SELECT=55 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:00:49.934100 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9420a36d-165d-4249-93cb-cb5625a8220a service cinder] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:49.936315 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-9420a36d-165d-4249-93cb-cb5625a8220a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:49.957985 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922586e9-aabe-4550-9a07-fa4fb4af2f94 service cinder] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:50.052071 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:00:50.109178 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9420a36d-165d-4249-93cb-cb5625a8220a service cinder] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:50.116361 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd4e895-cb38-420b-81bc-0696d425db53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:50.117775 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd4e895-cb38-420b-81bc-0696d425db53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:50.117960 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd4e895-cb38-420b-81bc-0696d425db53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:50.118350 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd4e895-cb38-420b-81bc-0696d425db53 admin admin] PATH_INFO: `/v3/users/a1cbbbbde3264f51b0038a7708056c6d` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:50.119635 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd4e895-cb38-420b-81bc-0696d425db53 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1cbbbbde3264f51b0038a7708056c6d)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:50.121279 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922586e9-aabe-4550-9a07-fa4fb4af2f94 service cinder] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:50.122680 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd4e895-cb38-420b-81bc-0696d425db53 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:50.133432 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab5e626-195b-4c73-853d-4d61619cd57f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:50.135727 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab5e626-195b-4c73-853d-4d61619cd57f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:50.136275 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab5e626-195b-4c73-853d-4d61619cd57f admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:50.136742 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab5e626-195b-4c73-853d-4d61619cd57f admin admin] PATH_INFO: `/v3/users/da5d687afab048a2addda468a586661e` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:50.140578 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab5e626-195b-4c73-853d-4d61619cd57f admin admin] RBAC: Authorizing `identity:delete_user(user_id=da5d687afab048a2addda468a586661e)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:50.144688 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab5e626-195b-4c73-853d-4d61619cd57f admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:50.200534 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-0cd4e895-cb38-420b-81bc-0696d425db53 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1cbbbbde3264f51b0038a7708056c6d'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:50.243339 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-0cd4e895-cb38-420b-81bc-0696d425db53 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1cbbbbde3264f51b0038a7708056c6d'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:50.247773 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-bab5e626-195b-4c73-853d-4d61619cd57f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da5d687afab048a2addda468a586661e'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:50.257330 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-0cd4e895-cb38-420b-81bc-0696d425db53 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1cbbbbde3264f51b0038a7708056c6d'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:50.258889 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-0cd4e895-cb38-420b-81bc-0696d425db53 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1cbbbbde3264f51b0038a7708056c6d'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:50.276053 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-0cd4e895-cb38-420b-81bc-0696d425db53 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1cbbbbde3264f51b0038a7708056c6d'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:50.276413 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-bab5e626-195b-4c73-853d-4d61619cd57f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da5d687afab048a2addda468a586661e'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:50.278158 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-bab5e626-195b-4c73-853d-4d61619cd57f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da5d687afab048a2addda468a586661e'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:50.286419 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 787/1576] 10.222.0.59 () {58 vars in 1224 bytes} [Tue Oct 11 12:00:49 2022] DELETE /identity/v3/users/a1cbbbbde3264f51b0038a7708056c6d => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:00:50.291587 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-bab5e626-195b-4c73-853d-4d61619cd57f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da5d687afab048a2addda468a586661e'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:50.292688 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-bab5e626-195b-4c73-853d-4d61619cd57f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da5d687afab048a2addda468a586661e'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:50.293124 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0eebaa7-0f6d-4da3-819e-7fab20bc238b None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:50.293488 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0eebaa7-0f6d-4da3-819e-7fab20bc238b None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:50.293899 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0eebaa7-0f6d-4da3-819e-7fab20bc238b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:50.314783 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 790/1577] 10.222.0.59 () {58 vars in 1224 bytes} [Tue Oct 11 12:00:49 2022] DELETE /identity/v3/users/da5d687afab048a2addda468a586661e => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:00:50.319222 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3f8196-8a06-4da4-90d0-b053aa4faf2f None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:50.319545 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3f8196-8a06-4da4-90d0-b053aa4faf2f None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:50.319812 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3f8196-8a06-4da4-90d0-b053aa4faf2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:50.322056 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-e0eebaa7-0f6d-4da3-819e-7fab20bc238b None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:00:50.323430 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-e0eebaa7-0f6d-4da3-819e-7fab20bc238b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:50.347367 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-7d3f8196-8a06-4da4-90d0-b053aa4faf2f None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:00:50.349036 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-7d3f8196-8a06-4da4-90d0-b053aa4faf2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:50.412276 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 788/1578] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:00:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:00:50.427295 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0eebaa7-0f6d-4da3-819e-7fab20bc238b None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:50.428788 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-e0eebaa7-0f6d-4da3-819e-7fab20bc238b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:50.438292 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 791/1579] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:00:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 12:00:50.445228 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3f8196-8a06-4da4-90d0-b053aa4faf2f None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:50.446877 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-7d3f8196-8a06-4da4-90d0-b053aa4faf2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:50.471097 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:00:50.530596 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0eebaa7-0f6d-4da3-819e-7fab20bc238b None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:50.535180 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09817d4-d850-448b-b4ee-74781423d2b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e563fe174db4cc19ae6bf8162cc3d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:50.536189 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09817d4-d850-448b-b4ee-74781423d2b1 service neutron] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:50.536539 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09817d4-d850-448b-b4ee-74781423d2b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:50.536879 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09817d4-d850-448b-b4ee-74781423d2b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:50.543096 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3f8196-8a06-4da4-90d0-b053aa4faf2f None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:50.548919 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347bcbe0-e35f-4d0c-8803-62764f7bbd9f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e563fe174db4cc19ae6bf8162cc3d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:50.549479 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09817d4-d850-448b-b4ee-74781423d2b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:50.550059 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347bcbe0-e35f-4d0c-8803-62764f7bbd9f service neutron] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:50.550500 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347bcbe0-e35f-4d0c-8803-62764f7bbd9f service neutron] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:50.550950 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347bcbe0-e35f-4d0c-8803-62764f7bbd9f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:50.551268 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09817d4-d850-448b-b4ee-74781423d2b1 service neutron] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:50.563198 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 789/1580] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:00:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:00:50.567946 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347bcbe0-e35f-4d0c-8803-62764f7bbd9f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:50.570807 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347bcbe0-e35f-4d0c-8803-62764f7bbd9f service neutron] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:50.578966 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:00:50.597250 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 792/1581] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:00:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 11 12:00:51.385617 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09817d4-d850-448b-b4ee-74781423d2b1 service neutron] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:51.388766 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-b09817d4-d850-448b-b4ee-74781423d2b1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:51.408346 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347bcbe0-e35f-4d0c-8803-62764f7bbd9f service neutron] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:51.412353 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-347bcbe0-e35f-4d0c-8803-62764f7bbd9f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:51.452714 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09817d4-d850-448b-b4ee-74781423d2b1 service neutron] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:51.482701 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28490502-6d0e-4fd9-b2df-863f26fba0a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:51.484386 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28490502-6d0e-4fd9-b2df-863f26fba0a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:51.486682 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28490502-6d0e-4fd9-b2df-863f26fba0a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:51.486682 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28490502-6d0e-4fd9-b2df-863f26fba0a2 admin admin] PATH_INFO: `/v3/projects/2e24b09232504f99be424f15212efeb2` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:51.488332 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347bcbe0-e35f-4d0c-8803-62764f7bbd9f service neutron] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:51.494211 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28490502-6d0e-4fd9-b2df-863f26fba0a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e24b09232504f99be424f15212efeb2)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:51.498063 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28490502-6d0e-4fd9-b2df-863f26fba0a2 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:51.522596 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e900ca68-2390-4393-b1fa-65acc6d8f4e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:51.523305 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e900ca68-2390-4393-b1fa-65acc6d8f4e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:51.523591 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e900ca68-2390-4393-b1fa-65acc6d8f4e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:51.523873 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e900ca68-2390-4393-b1fa-65acc6d8f4e2 admin admin] PATH_INFO: `/v3/projects/5a73174d8ceb48748d21449cd773364e` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:51.531833 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e900ca68-2390-4393-b1fa-65acc6d8f4e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a73174d8ceb48748d21449cd773364e)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:51.534324 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e900ca68-2390-4393-b1fa-65acc6d8f4e2 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:51.737994 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-28490502-6d0e-4fd9-b2df-863f26fba0a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e24b09232504f99be424f15212efeb2', 'request_id': 'req-28490502-6d0e-4fd9-b2df-863f26fba0a2'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:51.756459 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-e900ca68-2390-4393-b1fa-65acc6d8f4e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a73174d8ceb48748d21449cd773364e', 'request_id': 'req-e900ca68-2390-4393-b1fa-65acc6d8f4e2'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:51.759665 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-28490502-6d0e-4fd9-b2df-863f26fba0a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e24b09232504f99be424f15212efeb2', 'request_id': 'req-28490502-6d0e-4fd9-b2df-863f26fba0a2'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:51.774979 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-e900ca68-2390-4393-b1fa-65acc6d8f4e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a73174d8ceb48748d21449cd773364e', 'request_id': 'req-e900ca68-2390-4393-b1fa-65acc6d8f4e2'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:51.778203 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-28490502-6d0e-4fd9-b2df-863f26fba0a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e24b09232504f99be424f15212efeb2', 'request_id': 'req-28490502-6d0e-4fd9-b2df-863f26fba0a2'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:51.817266 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-28490502-6d0e-4fd9-b2df-863f26fba0a2 admin admin] The token cache is being invalidate because project 2e24b09232504f99be424f15212efeb2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 12:00:51.817588 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-28490502-6d0e-4fd9-b2df-863f26fba0a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:51.818790 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-28490502-6d0e-4fd9-b2df-863f26fba0a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:51.826498 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 790/1582] 10.222.0.59 () {58 vars in 1233 bytes} [Tue Oct 11 12:00:51 2022] DELETE /identity/v3/projects/2e24b09232504f99be424f15212efeb2 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:00:51.826758 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-e900ca68-2390-4393-b1fa-65acc6d8f4e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a73174d8ceb48748d21449cd773364e', 'request_id': 'req-e900ca68-2390-4393-b1fa-65acc6d8f4e2'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:51.828852 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28490502-6d0e-4fd9-b2df-863f26fba0a2 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:51.832775 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-28490502-6d0e-4fd9-b2df-863f26fba0a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:51.850183 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-e900ca68-2390-4393-b1fa-65acc6d8f4e2 admin admin] The token cache is being invalidate because project 5a73174d8ceb48748d21449cd773364e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81432) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 12:00:51.850809 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-e900ca68-2390-4393-b1fa-65acc6d8f4e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:51.852148 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-e900ca68-2390-4393-b1fa-65acc6d8f4e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:51.855299 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 793/1583] 10.222.0.59 () {58 vars in 1233 bytes} [Tue Oct 11 12:00:51 2022] DELETE /identity/v3/projects/5a73174d8ceb48748d21449cd773364e => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:00:51.864694 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e900ca68-2390-4393-b1fa-65acc6d8f4e2 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:51.865932 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-e900ca68-2390-4393-b1fa-65acc6d8f4e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:51.975730 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28490502-6d0e-4fd9-b2df-863f26fba0a2 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:52.036643 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e900ca68-2390-4393-b1fa-65acc6d8f4e2 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:52.047272 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12164994-c691-483b-a0f5-a7d849286014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:52.048420 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12164994-c691-483b-a0f5-a7d849286014 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:52.048901 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12164994-c691-483b-a0f5-a7d849286014 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:52.049421 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12164994-c691-483b-a0f5-a7d849286014 admin admin] PATH_INFO: `/v3/users/e6f1f89743a540e4acac7bdd0c6c139d` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:52.051671 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12164994-c691-483b-a0f5-a7d849286014 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6f1f89743a540e4acac7bdd0c6c139d)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:52.055958 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12164994-c691-483b-a0f5-a7d849286014 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:52.058852 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:00:52.080752 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8364d60-a5b2-4606-aeac-6380258b304a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:52.081980 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8364d60-a5b2-4606-aeac-6380258b304a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:52.082888 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8364d60-a5b2-4606-aeac-6380258b304a admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:52.083387 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8364d60-a5b2-4606-aeac-6380258b304a admin admin] PATH_INFO: `/v3/users/4ee624a12fad45da8a00a844ef136626` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:52.086722 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8364d60-a5b2-4606-aeac-6380258b304a admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ee624a12fad45da8a00a844ef136626)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:52.091898 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8364d60-a5b2-4606-aeac-6380258b304a admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:52.155077 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-12164994-c691-483b-a0f5-a7d849286014 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6f1f89743a540e4acac7bdd0c6c139d'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:52.180315 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-12164994-c691-483b-a0f5-a7d849286014 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6f1f89743a540e4acac7bdd0c6c139d'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:52.182246 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-12164994-c691-483b-a0f5-a7d849286014 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6f1f89743a540e4acac7bdd0c6c139d'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:52.191861 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-d8364d60-a5b2-4606-aeac-6380258b304a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ee624a12fad45da8a00a844ef136626'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:52.194966 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-12164994-c691-483b-a0f5-a7d849286014 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6f1f89743a540e4acac7bdd0c6c139d'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:52.196784 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-12164994-c691-483b-a0f5-a7d849286014 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6f1f89743a540e4acac7bdd0c6c139d'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:52.216392 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:00:52.217976 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-d8364d60-a5b2-4606-aeac-6380258b304a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ee624a12fad45da8a00a844ef136626'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:52.224806 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 794/1584] 10.222.0.59 () {58 vars in 1224 bytes} [Tue Oct 11 12:00:51 2022] DELETE /identity/v3/users/e6f1f89743a540e4acac7bdd0c6c139d => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:00:52.240524 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-d8364d60-a5b2-4606-aeac-6380258b304a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ee624a12fad45da8a00a844ef136626'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:52.241717 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-d8364d60-a5b2-4606-aeac-6380258b304a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ee624a12fad45da8a00a844ef136626'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:52.257738 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-d8364d60-a5b2-4606-aeac-6380258b304a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ee624a12fad45da8a00a844ef136626'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:52.270402 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 791/1585] 10.222.0.59 () {58 vars in 1224 bytes} [Tue Oct 11 12:00:51 2022] DELETE /identity/v3/users/4ee624a12fad45da8a00a844ef136626 => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 12:00:53.090542 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12164994-c691-483b-a0f5-a7d849286014 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:53.092692 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-12164994-c691-483b-a0f5-a7d849286014 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:53.239121 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12164994-c691-483b-a0f5-a7d849286014 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:53.334375 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33375d6-fc9f-4d77-b2e9-056647778a21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:53.337130 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33375d6-fc9f-4d77-b2e9-056647778a21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:53.337130 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33375d6-fc9f-4d77-b2e9-056647778a21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:53.337130 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33375d6-fc9f-4d77-b2e9-056647778a21 admin admin] PATH_INFO: `/v3/projects/54a3e98b3aaa421c9f2e3ef580921ac6` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:53.338136 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33375d6-fc9f-4d77-b2e9-056647778a21 admin admin] RBAC: Authorizing `identity:delete_project(project_id=54a3e98b3aaa421c9f2e3ef580921ac6)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:53.342392 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33375d6-fc9f-4d77-b2e9-056647778a21 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:53.433621 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-c33375d6-fc9f-4d77-b2e9-056647778a21 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '54a3e98b3aaa421c9f2e3ef580921ac6', 'request_id': 'req-c33375d6-fc9f-4d77-b2e9-056647778a21'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:53.442326 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-c33375d6-fc9f-4d77-b2e9-056647778a21 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '54a3e98b3aaa421c9f2e3ef580921ac6', 'request_id': 'req-c33375d6-fc9f-4d77-b2e9-056647778a21'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:53.454139 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-c33375d6-fc9f-4d77-b2e9-056647778a21 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '54a3e98b3aaa421c9f2e3ef580921ac6', 'request_id': 'req-c33375d6-fc9f-4d77-b2e9-056647778a21'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:53.477768 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8364d60-a5b2-4606-aeac-6380258b304a admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:53.482055 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-d8364d60-a5b2-4606-aeac-6380258b304a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:53.506259 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-c33375d6-fc9f-4d77-b2e9-056647778a21 admin admin] The token cache is being invalidate because project 54a3e98b3aaa421c9f2e3ef580921ac6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81432) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 12:00:53.507207 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-c33375d6-fc9f-4d77-b2e9-056647778a21 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:53.508924 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-c33375d6-fc9f-4d77-b2e9-056647778a21 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:53.512340 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 795/1586] 10.222.0.59 () {58 vars in 1233 bytes} [Tue Oct 11 12:00:53 2022] DELETE /identity/v3/projects/54a3e98b3aaa421c9f2e3ef580921ac6 => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:00:53.522382 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3a2e51-9b61-49f3-97d6-448d7f5c3eee None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:53.522841 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8364d60-a5b2-4606-aeac-6380258b304a admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:53.523693 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3a2e51-9b61-49f3-97d6-448d7f5c3eee None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:53.524698 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3a2e51-9b61-49f3-97d6-448d7f5c3eee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:53.569029 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-da3a2e51-9b61-49f3-97d6-448d7f5c3eee None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:00:53.577171 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-da3a2e51-9b61-49f3-97d6-448d7f5c3eee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:53.639637 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf76c3b-7763-41d1-98ac-7ad31235ed80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:53.640363 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf76c3b-7763-41d1-98ac-7ad31235ed80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:53.640649 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf76c3b-7763-41d1-98ac-7ad31235ed80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:53.640927 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf76c3b-7763-41d1-98ac-7ad31235ed80 admin admin] PATH_INFO: `/v3/projects/c3338a5037d74001ac918f972e019fc9` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:53.642600 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf76c3b-7763-41d1-98ac-7ad31235ed80 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3338a5037d74001ac918f972e019fc9)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:53.644693 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf76c3b-7763-41d1-98ac-7ad31235ed80 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:53.645630 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:00:53.691487 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 796/1587] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:00:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:00:53.702279 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e685ba2-d623-48f4-b76b-13a0cd5fa158 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:53.702723 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e685ba2-d623-48f4-b76b-13a0cd5fa158 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:53.703323 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e685ba2-d623-48f4-b76b-13a0cd5fa158 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:53.734955 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-4cf76c3b-7763-41d1-98ac-7ad31235ed80 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3338a5037d74001ac918f972e019fc9', 'request_id': 'req-4cf76c3b-7763-41d1-98ac-7ad31235ed80'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:53.742296 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-2e685ba2-d623-48f4-b76b-13a0cd5fa158 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:00:53.744430 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-4cf76c3b-7763-41d1-98ac-7ad31235ed80 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3338a5037d74001ac918f972e019fc9', 'request_id': 'req-4cf76c3b-7763-41d1-98ac-7ad31235ed80'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:53.747454 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-2e685ba2-d623-48f4-b76b-13a0cd5fa158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:53.762671 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-4cf76c3b-7763-41d1-98ac-7ad31235ed80 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3338a5037d74001ac918f972e019fc9', 'request_id': 'req-4cf76c3b-7763-41d1-98ac-7ad31235ed80'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:53.788987 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-4cf76c3b-7763-41d1-98ac-7ad31235ed80 admin admin] The token cache is being invalidate because project c3338a5037d74001ac918f972e019fc9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 12:00:53.789397 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-4cf76c3b-7763-41d1-98ac-7ad31235ed80 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:53.790298 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-4cf76c3b-7763-41d1-98ac-7ad31235ed80 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:53.794617 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 792/1588] 10.222.0.59 () {58 vars in 1233 bytes} [Tue Oct 11 12:00:53 2022] DELETE /identity/v3/projects/c3338a5037d74001ac918f972e019fc9 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:00:53.801843 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e84b72b-818a-43da-9067-927cc2e5c80a None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:53.803407 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e84b72b-818a-43da-9067-927cc2e5c80a None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:53.803725 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e84b72b-818a-43da-9067-927cc2e5c80a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:53.816062 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:00:53.832165 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-6e84b72b-818a-43da-9067-927cc2e5c80a None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:00:53.835946 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-6e84b72b-818a-43da-9067-927cc2e5c80a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:53.871670 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 797/1589] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:00:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:00:53.877404 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e685ba2-d623-48f4-b76b-13a0cd5fa158 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:53.893046 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e685ba2-d623-48f4-b76b-13a0cd5fa158 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:53.901360 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee934f3b-ae2a-4e50-9e15-b57e4cf6d6ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:53.903385 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee934f3b-ae2a-4e50-9e15-b57e4cf6d6ef admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:53.903818 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee934f3b-ae2a-4e50-9e15-b57e4cf6d6ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:53.904238 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee934f3b-ae2a-4e50-9e15-b57e4cf6d6ef admin admin] PATH_INFO: `/v3/domains` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:53.905485 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee934f3b-ae2a-4e50-9e15-b57e4cf6d6ef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 12:00:53.905772 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee934f3b-ae2a-4e50-9e15-b57e4cf6d6ef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:53.909160 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee934f3b-ae2a-4e50-9e15-b57e4cf6d6ef admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:53.946014 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 798/1590] 10.222.0.59 () {58 vars in 1153 bytes} [Tue Oct 11 12:00:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 12:00:53.954067 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee934f3b-ae2a-4e50-9e15-b57e4cf6d6ef admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:53.996747 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee934f3b-ae2a-4e50-9e15-b57e4cf6d6ef admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:54.004496 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 793/1591] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:00:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:00:54.009099 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2048dd6-112a-48f0-ab95-9324fccfe62e None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:54.009425 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2048dd6-112a-48f0-ab95-9324fccfe62e None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:54.009716 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2048dd6-112a-48f0-ab95-9324fccfe62e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:54.021331 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643629f0-30ff-41c1-b19d-4cedf91b55c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:54.021989 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643629f0-30ff-41c1-b19d-4cedf91b55c7 admin admin] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:54.022276 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643629f0-30ff-41c1-b19d-4cedf91b55c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:54.022584 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643629f0-30ff-41c1-b19d-4cedf91b55c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:54.023291 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643629f0-30ff-41c1-b19d-4cedf91b55c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:54.025784 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643629f0-30ff-41c1-b19d-4cedf91b55c7 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:54.039440 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-e2048dd6-112a-48f0-ab95-9324fccfe62e None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:00:54.041205 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-e2048dd6-112a-48f0-ab95-9324fccfe62e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:54.053934 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 794/1592] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:00:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:00:54.061764 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2048dd6-112a-48f0-ab95-9324fccfe62e None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:54.078344 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2048dd6-112a-48f0-ab95-9324fccfe62e None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:54.094302 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e86ef8-ae0a-4092-a6d0-6010254bf5a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:54.095434 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e86ef8-ae0a-4092-a6d0-6010254bf5a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:54.096050 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e86ef8-ae0a-4092-a6d0-6010254bf5a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:54.096464 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e86ef8-ae0a-4092-a6d0-6010254bf5a9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:54.097305 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e86ef8-ae0a-4092-a6d0-6010254bf5a9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 12:00:54.097858 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e86ef8-ae0a-4092-a6d0-6010254bf5a9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:54.101235 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e86ef8-ae0a-4092-a6d0-6010254bf5a9 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:54.102016 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 799/1593] 10.222.0.59 () {60 vars in 1153 bytes} [Tue Oct 11 12:00:53 2022] POST /identity/v3/projects => generated 377 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:00:54.104335 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643629f0-30ff-41c1-b19d-4cedf91b55c7 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:54.125711 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 795/1594] 10.222.0.59 () {58 vars in 1153 bytes} [Tue Oct 11 12:00:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 12:00:54.132518 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e86ef8-ae0a-4092-a6d0-6010254bf5a9 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:54.154428 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e86ef8-ae0a-4092-a6d0-6010254bf5a9 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:54.248201 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c85d74f-95cf-4718-9b00-b4c976cdd79e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:54.249161 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c85d74f-95cf-4718-9b00-b4c976cdd79e admin admin] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:54.249487 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c85d74f-95cf-4718-9b00-b4c976cdd79e admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:54.249766 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c85d74f-95cf-4718-9b00-b4c976cdd79e admin admin] PATH_INFO: `/v3/projects` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:54.250432 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c85d74f-95cf-4718-9b00-b4c976cdd79e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:54.253317 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c85d74f-95cf-4718-9b00-b4c976cdd79e admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:54.259922 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643629f0-30ff-41c1-b19d-4cedf91b55c7 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:54.267994 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3eaaeb1-1c52-439c-a2a8-247aa82ff44d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:54.269041 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eaaeb1-1c52-439c-a2a8-247aa82ff44d admin admin] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:54.269512 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eaaeb1-1c52-439c-a2a8-247aa82ff44d admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:54.270225 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eaaeb1-1c52-439c-a2a8-247aa82ff44d admin admin] PATH_INFO: `/v3/users` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:54.272810 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3eaaeb1-1c52-439c-a2a8-247aa82ff44d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:54.280952 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3eaaeb1-1c52-439c-a2a8-247aa82ff44d admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:54.294756 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 796/1595] 10.222.0.59 () {60 vars in 1153 bytes} [Tue Oct 11 12:00:54 2022] POST /identity/v3/projects => generated 409 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 12:00:54.300293 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c85d74f-95cf-4718-9b00-b4c976cdd79e admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:54.334098 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 800/1596] 10.222.0.59 () {60 vars in 1144 bytes} [Tue Oct 11 12:00:54 2022] POST /identity/v3/users => generated 289 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 12:00:54.339820 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3eaaeb1-1c52-439c-a2a8-247aa82ff44d admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:54.358733 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3eaaeb1-1c52-439c-a2a8-247aa82ff44d admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:54.408073 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c85d74f-95cf-4718-9b00-b4c976cdd79e admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:54.415797 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4041b7-3fb9-48b8-8e30-91a290cb4987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:54.418271 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4041b7-3fb9-48b8-8e30-91a290cb4987 admin admin] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:54.418271 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4041b7-3fb9-48b8-8e30-91a290cb4987 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:54.418271 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4041b7-3fb9-48b8-8e30-91a290cb4987 admin admin] PATH_INFO: `/v3/users` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:54.418960 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4041b7-3fb9-48b8-8e30-91a290cb4987 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:54.422197 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4041b7-3fb9-48b8-8e30-91a290cb4987 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:54.432773 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:00:54.450538 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:00:54.482029 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 797/1597] 10.222.0.59 () {60 vars in 1144 bytes} [Tue Oct 11 12:00:54 2022] POST /identity/v3/users => generated 305 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:00:54.483221 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4041b7-3fb9-48b8-8e30-91a290cb4987 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:54.503047 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4041b7-3fb9-48b8-8e30-91a290cb4987 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:54.518891 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df9fdb1-0db7-4fba-a4b5-f6ac986e2337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:54.519926 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df9fdb1-0db7-4fba-a4b5-f6ac986e2337 admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:54.520352 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df9fdb1-0db7-4fba-a4b5-f6ac986e2337 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:54.520701 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df9fdb1-0db7-4fba-a4b5-f6ac986e2337 admin admin] PATH_INFO: `/v3/roles` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:54.521608 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df9fdb1-0db7-4fba-a4b5-f6ac986e2337 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:54.523041 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f656af-46c3-4551-8f63-a2516a097ded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:54.523370 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df9fdb1-0db7-4fba-a4b5-f6ac986e2337 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:54.524252 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f656af-46c3-4551-8f63-a2516a097ded admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:54.524769 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f656af-46c3-4551-8f63-a2516a097ded admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:54.525311 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f656af-46c3-4551-8f63-a2516a097ded admin admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:54.526733 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f656af-46c3-4551-8f63-a2516a097ded admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:54.530974 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f656af-46c3-4551-8f63-a2516a097ded admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:54.549441 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 801/1598] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:00:54 2022] GET /identity/v3/roles => generated 1420 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:00:54.550857 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f656af-46c3-4551-8f63-a2516a097ded admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:54.561361 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 798/1599] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:00:54 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:00:54.568832 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df9fdb1-0db7-4fba-a4b5-f6ac986e2337 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:54.569071 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f656af-46c3-4551-8f63-a2516a097ded admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:54.602513 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031aa237-8142-4f17-898e-880d186f75fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:54.603195 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031aa237-8142-4f17-898e-880d186f75fc admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:54.603633 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df9fdb1-0db7-4fba-a4b5-f6ac986e2337 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:54.606925 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031aa237-8142-4f17-898e-880d186f75fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:54.607191 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031aa237-8142-4f17-898e-880d186f75fc admin admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:54.610032 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031aa237-8142-4f17-898e-880d186f75fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:54.617357 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ac3548-93a1-4db7-b84d-27bf3a2f1372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:54.619255 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ac3548-93a1-4db7-b84d-27bf3a2f1372 admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:54.619874 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ac3548-93a1-4db7-b84d-27bf3a2f1372 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:54.621392 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031aa237-8142-4f17-898e-880d186f75fc admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:54.621646 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ac3548-93a1-4db7-b84d-27bf3a2f1372 admin admin] PATH_INFO: `/v3/roles` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:54.623692 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ac3548-93a1-4db7-b84d-27bf3a2f1372 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:54.631547 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ac3548-93a1-4db7-b84d-27bf3a2f1372 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:54.646773 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 802/1600] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:00:54 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 11 12:00:54.651937 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031aa237-8142-4f17-898e-880d186f75fc admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:54.661012 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 799/1601] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:00:54 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 11 12:00:54.662278 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ac3548-93a1-4db7-b84d-27bf3a2f1372 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:54.672169 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031aa237-8142-4f17-898e-880d186f75fc admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:54.675852 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ac3548-93a1-4db7-b84d-27bf3a2f1372 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:54.680289 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f51e10c-c98a-4527-b776-8d7c271b8a4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:54.681360 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f51e10c-c98a-4527-b776-8d7c271b8a4c admin admin] REQUEST_METHOD: `PUT` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:54.681811 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f51e10c-c98a-4527-b776-8d7c271b8a4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:54.682395 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f51e10c-c98a-4527-b776-8d7c271b8a4c admin admin] PATH_INFO: `/v3/projects/1a802749244d43aeb29dfb2e029d2793/users/df4f56462f6d44d49a0cb547915d8df5/roles/de2d2bd26fff4748a79990f97a107164` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:54.683830 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6331d52c-4fe9-49ab-8a83-344cea28afa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:54.684662 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6331d52c-4fe9-49ab-8a83-344cea28afa1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:54.684943 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6331d52c-4fe9-49ab-8a83-344cea28afa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:54.685215 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6331d52c-4fe9-49ab-8a83-344cea28afa1 admin admin] PATH_INFO: `/v3/projects/67c854dbc3024ce28885a7eb854c8de4/users/8e8ff3ba800643b9bf478efdd101004b/roles/de2d2bd26fff4748a79990f97a107164` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:54.702201 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f51e10c-c98a-4527-b776-8d7c271b8a4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a802749244d43aeb29dfb2e029d2793, user_id=df4f56462f6d44d49a0cb547915d8df5, role_id=de2d2bd26fff4748a79990f97a107164)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:54.707253 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f51e10c-c98a-4527-b776-8d7c271b8a4c admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:54.710927 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6331d52c-4fe9-49ab-8a83-344cea28afa1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=67c854dbc3024ce28885a7eb854c8de4, user_id=8e8ff3ba800643b9bf478efdd101004b, role_id=de2d2bd26fff4748a79990f97a107164)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:54.716234 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6331d52c-4fe9-49ab-8a83-344cea28afa1 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:54.736833 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 803/1602] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 12:00:54 2022] PUT /identity/v3/projects/1a802749244d43aeb29dfb2e029d2793/users/df4f56462f6d44d49a0cb547915d8df5/roles/de2d2bd26fff4748a79990f97a107164 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:00:54.752805 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3c39f7-5956-4382-8e5c-86ab924aaecd None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:54.753138 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3c39f7-5956-4382-8e5c-86ab924aaecd None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:54.753375 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3c39f7-5956-4382-8e5c-86ab924aaecd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:54.791401 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 800/1603] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 12:00:54 2022] PUT /identity/v3/projects/67c854dbc3024ce28885a7eb854c8de4/users/8e8ff3ba800643b9bf478efdd101004b/roles/de2d2bd26fff4748a79990f97a107164 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:00:54.802773 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972c7f5c-bce2-46b7-a080-7a07c4222157 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:54.803103 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972c7f5c-bce2-46b7-a080-7a07c4222157 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:54.803318 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972c7f5c-bce2-46b7-a080-7a07c4222157 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:54.849053 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-972c7f5c-bce2-46b7-a080-7a07c4222157 None None] MFA Rules not processed for user `8e8ff3ba800643b9bf478efdd101004b`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:00:54.851932 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-972c7f5c-bce2-46b7-a080-7a07c4222157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:54.919145 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-2b3c39f7-5956-4382-8e5c-86ab924aaecd None None] MFA Rules not processed for user `df4f56462f6d44d49a0cb547915d8df5`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:00:54.921747 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-2b3c39f7-5956-4382-8e5c-86ab924aaecd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:54.981018 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 801/1604] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:00:54 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:00:54.982271 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972c7f5c-bce2-46b7-a080-7a07c4222157 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:55.045542 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 804/1605] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:00:54 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 12:00:55.052050 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3c39f7-5956-4382-8e5c-86ab924aaecd None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:55.091802 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972c7f5c-bce2-46b7-a080-7a07c4222157 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:55.101951 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7e4461-2a99-4294-9991-52fe6f17683a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:55.102780 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7e4461-2a99-4294-9991-52fe6f17683a admin admin] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:55.103062 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7e4461-2a99-4294-9991-52fe6f17683a admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:55.103331 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7e4461-2a99-4294-9991-52fe6f17683a admin admin] PATH_INFO: `/v3/projects` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:55.104168 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7e4461-2a99-4294-9991-52fe6f17683a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:55.111401 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7e4461-2a99-4294-9991-52fe6f17683a admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:55.167990 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3c39f7-5956-4382-8e5c-86ab924aaecd None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:55.174580 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d340764e-4fef-4abc-b247-7bf5878f0195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:55.175526 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d340764e-4fef-4abc-b247-7bf5878f0195 admin admin] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:55.175860 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d340764e-4fef-4abc-b247-7bf5878f0195 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:55.176167 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d340764e-4fef-4abc-b247-7bf5878f0195 admin admin] PATH_INFO: `/v3/projects` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:55.176866 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d340764e-4fef-4abc-b247-7bf5878f0195 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:55.178176 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 802/1606] 10.222.0.59 () {60 vars in 1153 bytes} [Tue Oct 11 12:00:54 2022] POST /identity/v3/projects => generated 407 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:00:55.179213 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7e4461-2a99-4294-9991-52fe6f17683a admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:55.180443 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d340764e-4fef-4abc-b247-7bf5878f0195 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:55.222314 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:00:55.240696 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 805/1607] 10.222.0.59 () {60 vars in 1153 bytes} [Tue Oct 11 12:00:55 2022] POST /identity/v3/projects => generated 379 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 12:00:55.253882 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d340764e-4fef-4abc-b247-7bf5878f0195 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:55.277534 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:00:55.321133 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7e4461-2a99-4294-9991-52fe6f17683a admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:55.385701 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d340764e-4fef-4abc-b247-7bf5878f0195 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:55.391216 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32805a65-7bfd-4522-9c1a-bd4960de6884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:55.391941 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32805a65-7bfd-4522-9c1a-bd4960de6884 admin admin] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:55.392220 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32805a65-7bfd-4522-9c1a-bd4960de6884 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:55.392493 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32805a65-7bfd-4522-9c1a-bd4960de6884 admin admin] PATH_INFO: `/v3/users` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:55.393362 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32805a65-7bfd-4522-9c1a-bd4960de6884 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:55.395496 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32805a65-7bfd-4522-9c1a-bd4960de6884 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:55.430531 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 806/1608] 10.222.0.59 () {60 vars in 1144 bytes} [Tue Oct 11 12:00:55 2022] POST /identity/v3/users => generated 290 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:00:55.431734 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32805a65-7bfd-4522-9c1a-bd4960de6884 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:55.433564 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-32805a65-7bfd-4522-9c1a-bd4960de6884 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:55.449993 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32805a65-7bfd-4522-9c1a-bd4960de6884 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:55.456975 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f5ce98-587d-410e-83d5-a92b634e1180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:55.457851 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f5ce98-587d-410e-83d5-a92b634e1180 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:55.458286 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f5ce98-587d-410e-83d5-a92b634e1180 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:55.458640 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f5ce98-587d-410e-83d5-a92b634e1180 admin admin] PATH_INFO: `/v3/users/c034ca927819468f8ad97b47ec1cbafd` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:55.460437 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f5ce98-587d-410e-83d5-a92b634e1180 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c034ca927819468f8ad97b47ec1cbafd)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:55.461772 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f24802d-4aed-4ef1-96c0-b77de81c7b97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:55.462608 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f5ce98-587d-410e-83d5-a92b634e1180 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:55.463186 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f24802d-4aed-4ef1-96c0-b77de81c7b97 admin admin] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:55.464152 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f24802d-4aed-4ef1-96c0-b77de81c7b97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:55.465055 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f24802d-4aed-4ef1-96c0-b77de81c7b97 admin admin] PATH_INFO: `/v3/users` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:55.467444 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f24802d-4aed-4ef1-96c0-b77de81c7b97 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:55.472276 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f24802d-4aed-4ef1-96c0-b77de81c7b97 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:55.562017 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 803/1609] 10.222.0.59 () {60 vars in 1144 bytes} [Tue Oct 11 12:00:55 2022] POST /identity/v3/users => generated 310 bytes in 383 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:00:55.566653 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-d5f5ce98-587d-410e-83d5-a92b634e1180 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c034ca927819468f8ad97b47ec1cbafd'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:55.569254 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f24802d-4aed-4ef1-96c0-b77de81c7b97 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:55.587461 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-d5f5ce98-587d-410e-83d5-a92b634e1180 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c034ca927819468f8ad97b47ec1cbafd'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:55.608339 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-d5f5ce98-587d-410e-83d5-a92b634e1180 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c034ca927819468f8ad97b47ec1cbafd'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:55.630048 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-d5f5ce98-587d-410e-83d5-a92b634e1180 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c034ca927819468f8ad97b47ec1cbafd'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:55.632227 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-d5f5ce98-587d-410e-83d5-a92b634e1180 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c034ca927819468f8ad97b47ec1cbafd'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:55.640433 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f24802d-4aed-4ef1-96c0-b77de81c7b97 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:55.659087 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fe9077-e5ea-42ae-84fb-550b6a2c0ec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:55.661193 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fe9077-e5ea-42ae-84fb-550b6a2c0ec0 admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:55.662264 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fe9077-e5ea-42ae-84fb-550b6a2c0ec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:55.663067 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fe9077-e5ea-42ae-84fb-550b6a2c0ec0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:55.664774 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fe9077-e5ea-42ae-84fb-550b6a2c0ec0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:55.668753 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fe9077-e5ea-42ae-84fb-550b6a2c0ec0 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:55.679291 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 807/1610] 10.222.0.59 () {58 vars in 1224 bytes} [Tue Oct 11 12:00:55 2022] DELETE /identity/v3/users/c034ca927819468f8ad97b47ec1cbafd => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:00:55.683406 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f5ce98-587d-410e-83d5-a92b634e1180 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:55.686793 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-d5f5ce98-587d-410e-83d5-a92b634e1180 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:55.696169 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 804/1611] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:00:55 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:00:55.699171 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b947391-cfd2-4a8f-9b49-b74acbcddd86 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:55.700045 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b947391-cfd2-4a8f-9b49-b74acbcddd86 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:55.702329 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b947391-cfd2-4a8f-9b49-b74acbcddd86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:55.748284 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-7b947391-cfd2-4a8f-9b49-b74acbcddd86 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:00:55.751223 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-7b947391-cfd2-4a8f-9b49-b74acbcddd86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:55.941554 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f5ce98-587d-410e-83d5-a92b634e1180 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:55.947519 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f432ca71-458b-4248-bf9d-71dea1904796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:55.948308 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f432ca71-458b-4248-bf9d-71dea1904796 admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:55.948747 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f432ca71-458b-4248-bf9d-71dea1904796 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:55.948885 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f432ca71-458b-4248-bf9d-71dea1904796 admin admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:55.949540 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f432ca71-458b-4248-bf9d-71dea1904796 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:55.951398 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f432ca71-458b-4248-bf9d-71dea1904796 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:55.968201 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 805/1612] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:00:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:00:55.969131 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b947391-cfd2-4a8f-9b49-b74acbcddd86 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:55.973779 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-7b947391-cfd2-4a8f-9b49-b74acbcddd86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:55.981309 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 808/1613] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:00:55 2022] GET /identity/v3/roles => generated 1420 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:00:55.984171 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f432ca71-458b-4248-bf9d-71dea1904796 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:55.997584 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b947391-cfd2-4a8f-9b49-b74acbcddd86 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:56.015462 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9f121d-c4af-4fbd-ad8f-5e113563117b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:56.016190 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9f121d-c4af-4fbd-ad8f-5e113563117b admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:56.016478 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9f121d-c4af-4fbd-ad8f-5e113563117b admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:56.016845 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9f121d-c4af-4fbd-ad8f-5e113563117b admin admin] PATH_INFO: `/v3/roles` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:56.017858 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9f121d-c4af-4fbd-ad8f-5e113563117b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:56.023120 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9f121d-c4af-4fbd-ad8f-5e113563117b admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:56.038376 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f432ca71-458b-4248-bf9d-71dea1904796 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:56.059778 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535864a0-c6df-44d7-9674-1213a4534770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:56.062317 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535864a0-c6df-44d7-9674-1213a4534770 admin admin] REQUEST_METHOD: `PUT` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:56.065315 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535864a0-c6df-44d7-9674-1213a4534770 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:56.067775 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535864a0-c6df-44d7-9674-1213a4534770 admin admin] PATH_INFO: `/v3/projects/b0a8be5046d3472e9d5efdee0931b887/users/54a7fd945cdb4032bcb771c5d0f390ac/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:56.073844 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 806/1614] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:00:55 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:00:56.091258 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:00:56.092757 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9f121d-c4af-4fbd-ad8f-5e113563117b admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:56.140066 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9f121d-c4af-4fbd-ad8f-5e113563117b admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:56.154052 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535864a0-c6df-44d7-9674-1213a4534770 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0a8be5046d3472e9d5efdee0931b887, user_id=54a7fd945cdb4032bcb771c5d0f390ac, role_id=5cf1f4bd2fef46ab9c6eff56d52f9f69)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:56.154676 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb36af1-f714-4b30-b5f3-143d517fcb5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:56.156108 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb36af1-f714-4b30-b5f3-143d517fcb5c admin admin] REQUEST_METHOD: `PUT` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:56.156689 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb36af1-f714-4b30-b5f3-143d517fcb5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:56.157269 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb36af1-f714-4b30-b5f3-143d517fcb5c admin admin] PATH_INFO: `/v3/projects/6a57fc9be1b74695a3d87e8d2f999296/users/8861e549cda14ce89b0dbe7f8e5bf808/roles/de2d2bd26fff4748a79990f97a107164` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:56.162598 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535864a0-c6df-44d7-9674-1213a4534770 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:56.198536 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb36af1-f714-4b30-b5f3-143d517fcb5c admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a57fc9be1b74695a3d87e8d2f999296, user_id=8861e549cda14ce89b0dbe7f8e5bf808, role_id=de2d2bd26fff4748a79990f97a107164)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:56.207063 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 809/1615] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 12:00:55 2022] PUT /identity/v3/projects/b0a8be5046d3472e9d5efdee0931b887/users/54a7fd945cdb4032bcb771c5d0f390ac/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:00:56.212729 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535864a0-c6df-44d7-9674-1213a4534770 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:56.216590 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-535864a0-c6df-44d7-9674-1213a4534770 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:56.219277 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb36af1-f714-4b30-b5f3-143d517fcb5c admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:56.274667 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 807/1616] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 12:00:56 2022] PUT /identity/v3/projects/6a57fc9be1b74695a3d87e8d2f999296/users/8861e549cda14ce89b0dbe7f8e5bf808/roles/de2d2bd26fff4748a79990f97a107164 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:00:56.275835 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb36af1-f714-4b30-b5f3-143d517fcb5c admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:56.363999 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:00:56.586294 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535864a0-c6df-44d7-9674-1213a4534770 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:56.669016 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb36af1-f714-4b30-b5f3-143d517fcb5c admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:56.679568 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a16f99-31a4-4937-a46d-50f0acbd6c6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:56.681761 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a16f99-31a4-4937-a46d-50f0acbd6c6f admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:56.682234 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a16f99-31a4-4937-a46d-50f0acbd6c6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:56.683644 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a16f99-31a4-4937-a46d-50f0acbd6c6f admin admin] PATH_INFO: `/v3/roles` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:56.684584 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a16f99-31a4-4937-a46d-50f0acbd6c6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:56.686909 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a16f99-31a4-4937-a46d-50f0acbd6c6f admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:56.706413 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 808/1617] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:00:56 2022] GET /identity/v3/roles => generated 1420 bytes in 435 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:00:56.707793 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73baec65-0df7-4b68-af61-19c21f0c9c1a None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:56.708721 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73baec65-0df7-4b68-af61-19c21f0c9c1a None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:56.708721 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73baec65-0df7-4b68-af61-19c21f0c9c1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:56.745934 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-73baec65-0df7-4b68-af61-19c21f0c9c1a None None] MFA Rules not processed for user `8861e549cda14ce89b0dbe7f8e5bf808`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:00:56.748232 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-73baec65-0df7-4b68-af61-19c21f0c9c1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:56.854493 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a6aad1-fe5f-4ed4-a837-93ed86c8d9d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e563fe174db4cc19ae6bf8162cc3d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:56.855617 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a6aad1-fe5f-4ed4-a837-93ed86c8d9d0 service neutron] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:56.856077 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a6aad1-fe5f-4ed4-a837-93ed86c8d9d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:56.856545 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a6aad1-fe5f-4ed4-a837-93ed86c8d9d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:56.869189 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 809/1618] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:00:56 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:00:56.870273 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73baec65-0df7-4b68-af61-19c21f0c9c1a None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:56.881182 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a6aad1-fe5f-4ed4-a837-93ed86c8d9d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:56.882825 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73baec65-0df7-4b68-af61-19c21f0c9c1a None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:56.884355 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a6aad1-fe5f-4ed4-a837-93ed86c8d9d0 service neutron] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:56.888055 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8c41de-2581-47f5-bac3-03ed9b6e75c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:56.888888 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8c41de-2581-47f5-bac3-03ed9b6e75c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:56.889251 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8c41de-2581-47f5-bac3-03ed9b6e75c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:56.889625 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8c41de-2581-47f5-bac3-03ed9b6e75c4 admin admin] PATH_INFO: `/v3/projects/b0a8be5046d3472e9d5efdee0931b887/users/54a7fd945cdb4032bcb771c5d0f390ac/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:56.891815 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8c41de-2581-47f5-bac3-03ed9b6e75c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0a8be5046d3472e9d5efdee0931b887, user_id=54a7fd945cdb4032bcb771c5d0f390ac, role_id=5cf1f4bd2fef46ab9c6eff56d52f9f69)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:56.894851 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8c41de-2581-47f5-bac3-03ed9b6e75c4 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:56.905969 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 810/1619] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:00:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 698 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:00:56.908039 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a6aad1-fe5f-4ed4-a837-93ed86c8d9d0 service neutron] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:56.908963 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 810/1620] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 12:00:56 2022] PUT /identity/v3/projects/b0a8be5046d3472e9d5efdee0931b887/users/54a7fd945cdb4032bcb771c5d0f390ac/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:00:56.929914 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0706e3b-9bf4-4f43-b7a7-7395314acde8 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:56.929914 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0706e3b-9bf4-4f43-b7a7-7395314acde8 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:56.930398 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0706e3b-9bf4-4f43-b7a7-7395314acde8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:56.944240 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a6aad1-fe5f-4ed4-a837-93ed86c8d9d0 service neutron] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:57.011991 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-d0706e3b-9bf4-4f43-b7a7-7395314acde8 None None] MFA Rules not processed for user `54a7fd945cdb4032bcb771c5d0f390ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:00:57.013767 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-d0706e3b-9bf4-4f43-b7a7-7395314acde8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:57.123113 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16877fee-cc18-4f17-bb04-08d299ffca0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:57.128786 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16877fee-cc18-4f17-bb04-08d299ffca0a admin admin] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:57.130299 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16877fee-cc18-4f17-bb04-08d299ffca0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:57.133116 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16877fee-cc18-4f17-bb04-08d299ffca0a admin admin] PATH_INFO: `/v3/projects` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:57.134882 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16877fee-cc18-4f17-bb04-08d299ffca0a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:57.148749 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16877fee-cc18-4f17-bb04-08d299ffca0a admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:57.203766 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 811/1621] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:00:56 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 12:00:57.203766 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8442cd8-3f6a-491d-b8f7-6a879cfc3f3a None None] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:57.203766 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8442cd8-3f6a-491d-b8f7-6a879cfc3f3a None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:57.203766 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8442cd8-3f6a-491d-b8f7-6a879cfc3f3a None None] PATH_INFO: `/` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:57.212216 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 812/1622] 10.222.0.59 () {54 vars in 921 bytes} [Tue Oct 11 12:00:57 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 12:00:57.213650 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4623d767-0069-43da-a17c-e470a4752871 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:57.213893 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4623d767-0069-43da-a17c-e470a4752871 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:57.214157 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4623d767-0069-43da-a17c-e470a4752871 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:57.234962 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 811/1623] 10.222.0.59 () {60 vars in 1153 bytes} [Tue Oct 11 12:00:56 2022] POST /identity/v3/projects => generated 375 bytes in 328 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:00:57.236860 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16877fee-cc18-4f17-bb04-08d299ffca0a admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:57.241324 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-16877fee-cc18-4f17-bb04-08d299ffca0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:57.254831 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-4623d767-0069-43da-a17c-e470a4752871 None None] MFA Rules not processed for user `bad5e1cb306b46818096940acc3810b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:00:57.255876 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-4623d767-0069-43da-a17c-e470a4752871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:57.271292 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:00:57.344381 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:00:57.371973 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 813/1624] 10.222.0.59 () {58 vars in 1018 bytes} [Tue Oct 11 12:00:57 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:00:57.386734 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4623d767-0069-43da-a17c-e470a4752871 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:57.402798 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16877fee-cc18-4f17-bb04-08d299ffca0a admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:57.409487 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd6ab77-63c6-4c21-98d2-907e07b11791 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b1b651c73664d08accc6edf2f79bace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:57.410292 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd6ab77-63c6-4c21-98d2-907e07b11791 service cinder] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:57.410747 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd6ab77-63c6-4c21-98d2-907e07b11791 service cinder] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:57.411131 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd6ab77-63c6-4c21-98d2-907e07b11791 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:57.482112 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4623d767-0069-43da-a17c-e470a4752871 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:57.488160 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8582a691-e45d-4f27-ac25-b12f210edb54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:57.488815 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8582a691-e45d-4f27-ac25-b12f210edb54 admin admin] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:57.489093 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8582a691-e45d-4f27-ac25-b12f210edb54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:57.489365 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8582a691-e45d-4f27-ac25-b12f210edb54 admin admin] PATH_INFO: `/v3/users` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:57.491077 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8582a691-e45d-4f27-ac25-b12f210edb54 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:57.493200 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8582a691-e45d-4f27-ac25-b12f210edb54 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:57.519430 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 814/1625] 10.222.0.59 () {60 vars in 1144 bytes} [Tue Oct 11 12:00:57 2022] POST /identity/v3/users => generated 294 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:00:57.520181 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8582a691-e45d-4f27-ac25-b12f210edb54 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:57.522612 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-8582a691-e45d-4f27-ac25-b12f210edb54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:57.541793 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8582a691-e45d-4f27-ac25-b12f210edb54 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:57.548239 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd6ab77-63c6-4c21-98d2-907e07b11791 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:57.549857 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd6ab77-63c6-4c21-98d2-907e07b11791 service cinder] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:57.552494 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7150613a-a5ef-46ed-a77b-783d0c77c9c0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bad5e1cb306b46818096940acc3810b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:57.556195 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7150613a-a5ef-46ed-a77b-783d0c77c9c0 service nova] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:57.556706 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7150613a-a5ef-46ed-a77b-783d0c77c9c0 service nova] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:57.557192 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7150613a-a5ef-46ed-a77b-783d0c77c9c0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:57.569190 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 812/1626] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:00:57 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:00:57.570205 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd6ab77-63c6-4c21-98d2-907e07b11791 service cinder] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:57.581452 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7150613a-a5ef-46ed-a77b-783d0c77c9c0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:57.584141 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd6ab77-63c6-4c21-98d2-907e07b11791 service cinder] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:57.584648 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7150613a-a5ef-46ed-a77b-783d0c77c9c0 service nova] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:57.589856 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4829c4-46e8-4606-b395-cd83cc15d19d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:57.590680 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4829c4-46e8-4606-b395-cd83cc15d19d admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:57.591012 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4829c4-46e8-4606-b395-cd83cc15d19d admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:57.591730 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4829c4-46e8-4606-b395-cd83cc15d19d admin admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:57.593015 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4829c4-46e8-4606-b395-cd83cc15d19d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:57.595956 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4829c4-46e8-4606-b395-cd83cc15d19d admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:57.602133 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 815/1627] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:00:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:00:57.612758 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 813/1628] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:00:57 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:00:57.618869 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7150613a-a5ef-46ed-a77b-783d0c77c9c0 service nova] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:57.637416 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7150613a-a5ef-46ed-a77b-783d0c77c9c0 service nova] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:57.648041 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92affb9-0597-4f24-a057-fefb09314fdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:57.649275 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92affb9-0597-4f24-a057-fefb09314fdd admin admin] REQUEST_METHOD: `PUT` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:57.650235 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92affb9-0597-4f24-a057-fefb09314fdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:57.650368 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92affb9-0597-4f24-a057-fefb09314fdd admin admin] PATH_INFO: `/v3/projects/440847454347404aa5604cc949ebabe5/users/bafe8313685e462b94055bcd6bbd5259/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:57.672988 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92affb9-0597-4f24-a057-fefb09314fdd admin admin] RBAC: Authorizing `identity:create_grant(project_id=440847454347404aa5604cc949ebabe5, user_id=bafe8313685e462b94055bcd6bbd5259, role_id=5cf1f4bd2fef46ab9c6eff56d52f9f69)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:57.676919 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92affb9-0597-4f24-a057-fefb09314fdd admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:57.696473 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 816/1629] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 12:00:57 2022] PUT /identity/v3/projects/440847454347404aa5604cc949ebabe5/users/bafe8313685e462b94055bcd6bbd5259/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 12:00:57.702749 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4829c4-46e8-4606-b395-cd83cc15d19d admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:57.724128 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4829c4-46e8-4606-b395-cd83cc15d19d admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:57.827876 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92affb9-0597-4f24-a057-fefb09314fdd admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:57.831137 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-a92affb9-0597-4f24-a057-fefb09314fdd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:57.901263 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abda4c06-b612-4c83-a8fb-3ff7dc0432a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:57.902891 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abda4c06-b612-4c83-a8fb-3ff7dc0432a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:57.903350 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abda4c06-b612-4c83-a8fb-3ff7dc0432a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:57.903764 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abda4c06-b612-4c83-a8fb-3ff7dc0432a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:57.905855 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abda4c06-b612-4c83-a8fb-3ff7dc0432a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:57.908989 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abda4c06-b612-4c83-a8fb-3ff7dc0432a0 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:57.944059 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 814/1630] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:00:57 2022] GET /identity/v3/roles => generated 1420 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:00:57.956296 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abda4c06-b612-4c83-a8fb-3ff7dc0432a0 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:57.983064 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abda4c06-b612-4c83-a8fb-3ff7dc0432a0 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:57.992321 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b4b649-7fa6-4903-b50f-48391e25dcf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:57.993146 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b4b649-7fa6-4903-b50f-48391e25dcf2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:57.994190 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b4b649-7fa6-4903-b50f-48391e25dcf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:57.995450 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b4b649-7fa6-4903-b50f-48391e25dcf2 admin admin] PATH_INFO: `/v3/projects/440847454347404aa5604cc949ebabe5/users/bafe8313685e462b94055bcd6bbd5259/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:57.998677 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b4b649-7fa6-4903-b50f-48391e25dcf2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=440847454347404aa5604cc949ebabe5, user_id=bafe8313685e462b94055bcd6bbd5259, role_id=5cf1f4bd2fef46ab9c6eff56d52f9f69)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:58.004664 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b4b649-7fa6-4903-b50f-48391e25dcf2 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:58.026212 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92affb9-0597-4f24-a057-fefb09314fdd admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:58.028658 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 815/1631] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 12:00:57 2022] PUT /identity/v3/projects/440847454347404aa5604cc949ebabe5/users/bafe8313685e462b94055bcd6bbd5259/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:00:58.029770 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b4b649-7fa6-4903-b50f-48391e25dcf2 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:58.052562 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:00:58.186157 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:00:58.199374 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b4b649-7fa6-4903-b50f-48391e25dcf2 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:58.205702 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f322a132-58ba-429f-8bff-b5fbcca572ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:58.207305 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f322a132-58ba-429f-8bff-b5fbcca572ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:58.208547 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f322a132-58ba-429f-8bff-b5fbcca572ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:58.209249 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f322a132-58ba-429f-8bff-b5fbcca572ee admin admin] PATH_INFO: `/v3/projects/d5a16d2bddcf4457b1bda9239ac43f9a` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:58.210138 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dff0df-1bdb-4326-97d1-ee9d63b91ea6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b1b651c73664d08accc6edf2f79bace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:58.211662 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dff0df-1bdb-4326-97d1-ee9d63b91ea6 service cinder] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:58.212290 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dff0df-1bdb-4326-97d1-ee9d63b91ea6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:58.212790 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f322a132-58ba-429f-8bff-b5fbcca572ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5a16d2bddcf4457b1bda9239ac43f9a)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:58.213118 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dff0df-1bdb-4326-97d1-ee9d63b91ea6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:58.216812 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-02dff0df-1bdb-4326-97d1-ee9d63b91ea6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:58.217049 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f322a132-58ba-429f-8bff-b5fbcca572ee admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:58.338782 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-f322a132-58ba-429f-8bff-b5fbcca572ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5a16d2bddcf4457b1bda9239ac43f9a', 'request_id': 'req-f322a132-58ba-429f-8bff-b5fbcca572ee'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:58.354228 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-f322a132-58ba-429f-8bff-b5fbcca572ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5a16d2bddcf4457b1bda9239ac43f9a', 'request_id': 'req-f322a132-58ba-429f-8bff-b5fbcca572ee'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:58.371927 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-f322a132-58ba-429f-8bff-b5fbcca572ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5a16d2bddcf4457b1bda9239ac43f9a', 'request_id': 'req-f322a132-58ba-429f-8bff-b5fbcca572ee'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:58.377414 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dff0df-1bdb-4326-97d1-ee9d63b91ea6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:58.378731 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dff0df-1bdb-4326-97d1-ee9d63b91ea6 service cinder] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:58.397738 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 816/1632] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:00:58 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:00:58.400841 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a24c31-3023-40c9-bd95-b239746cf4fc None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:58.401165 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a24c31-3023-40c9-bd95-b239746cf4fc None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:58.401393 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a24c31-3023-40c9-bd95-b239746cf4fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:58.409941 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-f322a132-58ba-429f-8bff-b5fbcca572ee admin admin] The token cache is being invalidate because project d5a16d2bddcf4457b1bda9239ac43f9a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 12:00:58.411508 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-f322a132-58ba-429f-8bff-b5fbcca572ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:58.414449 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-f322a132-58ba-429f-8bff-b5fbcca572ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:00:58.418676 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 817/1633] 10.222.0.59 () {58 vars in 1233 bytes} [Tue Oct 11 12:00:57 2022] DELETE /identity/v3/projects/d5a16d2bddcf4457b1bda9239ac43f9a => generated 0 bytes in 592 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:00:58.424086 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67fd6b9-bca9-4d51-8283-de56cc8f3229 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:58.424369 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67fd6b9-bca9-4d51-8283-de56cc8f3229 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:58.424635 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67fd6b9-bca9-4d51-8283-de56cc8f3229 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:58.482606 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-e67fd6b9-bca9-4d51-8283-de56cc8f3229 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:00:58.483960 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-e67fd6b9-bca9-4d51-8283-de56cc8f3229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:58.484781 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-86a24c31-3023-40c9-bd95-b239746cf4fc None None] MFA Rules not processed for user `bafe8313685e462b94055bcd6bbd5259`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:00:58.486057 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-86a24c31-3023-40c9-bd95-b239746cf4fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:58.698425 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 818/1634] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:00:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 12:00:58.707193 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd10b29d-4bb8-4bc3-8bad-febd55dcbae3 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:58.707673 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd10b29d-4bb8-4bc3-8bad-febd55dcbae3 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:58.708100 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd10b29d-4bb8-4bc3-8bad-febd55dcbae3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:58.720257 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 817/1635] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:00:58 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 322 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:00:58.743608 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-dd10b29d-4bb8-4bc3-8bad-febd55dcbae3 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:00:58.745756 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-dd10b29d-4bb8-4bc3-8bad-febd55dcbae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:58.756438 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 819/1636] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:00:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 12:00:58.763508 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a24c31-3023-40c9-bd95-b239746cf4fc None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:58.783883 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a24c31-3023-40c9-bd95-b239746cf4fc None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:58.793847 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac49a45a-e175-4ff8-81ea-0b21687543cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:58.795165 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac49a45a-e175-4ff8-81ea-0b21687543cd admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:58.795930 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac49a45a-e175-4ff8-81ea-0b21687543cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:58.796553 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac49a45a-e175-4ff8-81ea-0b21687543cd admin admin] PATH_INFO: `/v3/domains` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:58.797921 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac49a45a-e175-4ff8-81ea-0b21687543cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 12:00:58.798702 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac49a45a-e175-4ff8-81ea-0b21687543cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:58.803898 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac49a45a-e175-4ff8-81ea-0b21687543cd admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:58.837569 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 818/1637] 10.222.0.59 () {58 vars in 1153 bytes} [Tue Oct 11 12:00:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 12:00:58.843372 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd10b29d-4bb8-4bc3-8bad-febd55dcbae3 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:58.866674 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd10b29d-4bb8-4bc3-8bad-febd55dcbae3 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:58.876006 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a8b4a4-6ec6-43b0-b38f-6e3e0054ec40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:58.877276 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a8b4a4-6ec6-43b0-b38f-6e3e0054ec40 admin admin] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:58.877776 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a8b4a4-6ec6-43b0-b38f-6e3e0054ec40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:58.878253 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a8b4a4-6ec6-43b0-b38f-6e3e0054ec40 admin admin] PATH_INFO: `/v3/projects` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:58.879726 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a8b4a4-6ec6-43b0-b38f-6e3e0054ec40 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:58.883695 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a8b4a4-6ec6-43b0-b38f-6e3e0054ec40 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:58.928486 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 820/1638] 10.222.0.59 () {60 vars in 1153 bytes} [Tue Oct 11 12:00:58 2022] POST /identity/v3/projects => generated 363 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:00:58.932960 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac49a45a-e175-4ff8-81ea-0b21687543cd admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:58.944164 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac49a45a-e175-4ff8-81ea-0b21687543cd admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:59.047247 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a8b4a4-6ec6-43b0-b38f-6e3e0054ec40 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:59.050386 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-f9a8b4a4-6ec6-43b0-b38f-6e3e0054ec40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:59.081614 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:00:59.118704 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd560ce-3968-4a9c-9410-138b7aedd4e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:59.120084 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd560ce-3968-4a9c-9410-138b7aedd4e4 admin admin] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:59.134823 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd560ce-3968-4a9c-9410-138b7aedd4e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:59.134823 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd560ce-3968-4a9c-9410-138b7aedd4e4 admin admin] PATH_INFO: `/v3/users` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:59.134823 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd560ce-3968-4a9c-9410-138b7aedd4e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:59.134823 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd560ce-3968-4a9c-9410-138b7aedd4e4 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:59.246393 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 819/1639] 10.222.0.59 () {60 vars in 1144 bytes} [Tue Oct 11 12:00:58 2022] POST /identity/v3/users => generated 282 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:00:59.247544 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd560ce-3968-4a9c-9410-138b7aedd4e4 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:59.261052 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a8b4a4-6ec6-43b0-b38f-6e3e0054ec40 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:59.262200 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:00:59.271668 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd560ce-3968-4a9c-9410-138b7aedd4e4 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:59.279524 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5bf311-89c3-4230-bc85-8f64246dc0e2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b1b651c73664d08accc6edf2f79bace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:59.279898 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578569ee-906d-4413-882d-2de7c575905b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:59.280306 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578569ee-906d-4413-882d-2de7c575905b admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:59.280583 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578569ee-906d-4413-882d-2de7c575905b admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:59.280886 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578569ee-906d-4413-882d-2de7c575905b admin admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:59.281737 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578569ee-906d-4413-882d-2de7c575905b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:59.283121 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5bf311-89c3-4230-bc85-8f64246dc0e2 service cinder] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:59.283584 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578569ee-906d-4413-882d-2de7c575905b admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:59.284439 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5bf311-89c3-4230-bc85-8f64246dc0e2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:59.285185 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5bf311-89c3-4230-bc85-8f64246dc0e2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:59.311050 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 820/1640] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:00:59 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:00:59.318485 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578569ee-906d-4413-882d-2de7c575905b admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:59.345058 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578569ee-906d-4413-882d-2de7c575905b admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:59.363289 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237f6493-3f59-4978-b425-7008aa64b4ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:59.364428 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237f6493-3f59-4978-b425-7008aa64b4ef admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:59.369789 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237f6493-3f59-4978-b425-7008aa64b4ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:59.370503 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237f6493-3f59-4978-b425-7008aa64b4ef admin admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:59.373795 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237f6493-3f59-4978-b425-7008aa64b4ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:59.386978 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237f6493-3f59-4978-b425-7008aa64b4ef admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:59.423432 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 821/1641] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:00:59 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:00:59.425112 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237f6493-3f59-4978-b425-7008aa64b4ef admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:59.429299 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5bf311-89c3-4230-bc85-8f64246dc0e2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:59.443173 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5bf311-89c3-4230-bc85-8f64246dc0e2 service cinder] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:59.445959 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237f6493-3f59-4978-b425-7008aa64b4ef admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:59.464988 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ad5609-c915-4de7-a6e3-3327b9c84a9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:59.465708 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ad5609-c915-4de7-a6e3-3327b9c84a9d admin admin] REQUEST_METHOD: `PUT` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:59.466003 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ad5609-c915-4de7-a6e3-3327b9c84a9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:59.466278 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ad5609-c915-4de7-a6e3-3327b9c84a9d admin admin] PATH_INFO: `/v3/projects/caa7ebc594274b5c90fbd61e4b9f484a/users/fd5407c77f504c8aa832274eede4107a/roles/de2d2bd26fff4748a79990f97a107164` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:59.485950 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 821/1642] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:00:59 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:00:59.514220 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ad5609-c915-4de7-a6e3-3327b9c84a9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=caa7ebc594274b5c90fbd61e4b9f484a, user_id=fd5407c77f504c8aa832274eede4107a, role_id=de2d2bd26fff4748a79990f97a107164)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:59.518243 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ad5609-c915-4de7-a6e3-3327b9c84a9d admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:59.602190 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 822/1643] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 12:00:59 2022] PUT /identity/v3/projects/caa7ebc594274b5c90fbd61e4b9f484a/users/fd5407c77f504c8aa832274eede4107a/roles/de2d2bd26fff4748a79990f97a107164 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:00:59.617808 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5bf311-89c3-4230-bc85-8f64246dc0e2 service cinder] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:59.628058 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164a2d66-bac3-4226-a77b-b997b53c1cc6 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:59.630079 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164a2d66-bac3-4226-a77b-b997b53c1cc6 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:59.630866 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164a2d66-bac3-4226-a77b-b997b53c1cc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:59.642684 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5bf311-89c3-4230-bc85-8f64246dc0e2 service cinder] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:00:59.710174 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-164a2d66-bac3-4226-a77b-b997b53c1cc6 None None] MFA Rules not processed for user `fd5407c77f504c8aa832274eede4107a`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:00:59.712775 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-164a2d66-bac3-4226-a77b-b997b53c1cc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:59.792050 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ff9bb2-24bd-4d19-8878-e7de0a4b8bb0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b1b651c73664d08accc6edf2f79bace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:00:59.793720 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ff9bb2-24bd-4d19-8878-e7de0a4b8bb0 service cinder] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:00:59.794619 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ff9bb2-24bd-4d19-8878-e7de0a4b8bb0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:00:59.795223 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ff9bb2-24bd-4d19-8878-e7de0a4b8bb0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:00:59.798392 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-08ff9bb2-24bd-4d19-8878-e7de0a4b8bb0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:00:59.859335 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 823/1644] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:00:59 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:00:59.864389 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164a2d66-bac3-4226-a77b-b997b53c1cc6 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:00:59.948081 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ff9bb2-24bd-4d19-8878-e7de0a4b8bb0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:00:59.951371 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ff9bb2-24bd-4d19-8878-e7de0a4b8bb0 service cinder] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:00:59.984565 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 822/1645] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:00:59 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:01:00.046662 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164a2d66-bac3-4226-a77b-b997b53c1cc6 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:00.055330 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6290e07-38a9-4757-a86a-3a7360d78c53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:00.056116 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6290e07-38a9-4757-a86a-3a7360d78c53 admin admin] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:00.056461 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6290e07-38a9-4757-a86a-3a7360d78c53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:00.056810 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6290e07-38a9-4757-a86a-3a7360d78c53 admin admin] PATH_INFO: `/v3/projects` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:00.057547 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6290e07-38a9-4757-a86a-3a7360d78c53 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:00.061040 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6290e07-38a9-4757-a86a-3a7360d78c53 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:00.102963 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 824/1646] 10.222.0.59 () {60 vars in 1153 bytes} [Tue Oct 11 12:00:59 2022] POST /identity/v3/projects => generated 365 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:01:00.115299 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ff9bb2-24bd-4d19-8878-e7de0a4b8bb0 service cinder] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:00.133614 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ff9bb2-24bd-4d19-8878-e7de0a4b8bb0 service cinder] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:00.157694 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:01:00.255569 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7185011e-b7d4-45a8-812e-ee011da236b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:00.256348 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7185011e-b7d4-45a8-812e-ee011da236b8 admin admin] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:00.256765 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7185011e-b7d4-45a8-812e-ee011da236b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:00.257158 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7185011e-b7d4-45a8-812e-ee011da236b8 admin admin] PATH_INFO: `/v3/users` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:00.257906 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7185011e-b7d4-45a8-812e-ee011da236b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:00.260228 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7185011e-b7d4-45a8-812e-ee011da236b8 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:00.306469 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6290e07-38a9-4757-a86a-3a7360d78c53 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:00.313329 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 823/1647] 10.222.0.59 () {60 vars in 1144 bytes} [Tue Oct 11 12:01:00 2022] POST /identity/v3/users => generated 289 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:01:00.330791 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6290e07-38a9-4757-a86a-3a7360d78c53 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:00.340856 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525e7e17-857a-4603-a900-d5cc3963ef74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:00.341847 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525e7e17-857a-4603-a900-d5cc3963ef74 admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:00.342303 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525e7e17-857a-4603-a900-d5cc3963ef74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:00.342855 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525e7e17-857a-4603-a900-d5cc3963ef74 admin admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:00.344241 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525e7e17-857a-4603-a900-d5cc3963ef74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:00.349245 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525e7e17-857a-4603-a900-d5cc3963ef74 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:00.374639 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 825/1648] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:01:00 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:01:00.379234 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7185011e-b7d4-45a8-812e-ee011da236b8 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:00.399896 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7185011e-b7d4-45a8-812e-ee011da236b8 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:00.424624 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388bcbef-2f59-4b03-9b0c-bbbc0785f754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:00.425662 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388bcbef-2f59-4b03-9b0c-bbbc0785f754 admin admin] REQUEST_METHOD: `PUT` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:00.426073 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388bcbef-2f59-4b03-9b0c-bbbc0785f754 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:00.426476 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388bcbef-2f59-4b03-9b0c-bbbc0785f754 admin admin] PATH_INFO: `/v3/projects/0d5d64912f024fe4aaff912f85bd10f4/users/ce7b46d961a3401db525c8913397f81b/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:00.450848 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388bcbef-2f59-4b03-9b0c-bbbc0785f754 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d5d64912f024fe4aaff912f85bd10f4, user_id=ce7b46d961a3401db525c8913397f81b, role_id=5cf1f4bd2fef46ab9c6eff56d52f9f69)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:00.454358 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388bcbef-2f59-4b03-9b0c-bbbc0785f754 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:00.473468 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 824/1649] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 12:01:00 2022] PUT /identity/v3/projects/0d5d64912f024fe4aaff912f85bd10f4/users/ce7b46d961a3401db525c8913397f81b/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:01:00.478415 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525e7e17-857a-4603-a900-d5cc3963ef74 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:00.493965 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525e7e17-857a-4603-a900-d5cc3963ef74 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:00.548503 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:01:00.651926 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0760bb5a-16ce-4723-9425-4f25ebeddf9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:00.653256 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0760bb5a-16ce-4723-9425-4f25ebeddf9a admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:00.653792 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0760bb5a-16ce-4723-9425-4f25ebeddf9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:00.654277 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0760bb5a-16ce-4723-9425-4f25ebeddf9a admin admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:00.655358 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0760bb5a-16ce-4723-9425-4f25ebeddf9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:00.657349 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0760bb5a-16ce-4723-9425-4f25ebeddf9a admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:00.709580 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 826/1650] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:01:00 2022] GET /identity/v3/roles => generated 1420 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:01:00.714162 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388bcbef-2f59-4b03-9b0c-bbbc0785f754 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:00.733697 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388bcbef-2f59-4b03-9b0c-bbbc0785f754 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:00.758313 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c5395e-6082-4724-b304-599b00bfa62c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:00.762624 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c5395e-6082-4724-b304-599b00bfa62c admin admin] REQUEST_METHOD: `PUT` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:00.763193 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c5395e-6082-4724-b304-599b00bfa62c admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:00.763608 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c5395e-6082-4724-b304-599b00bfa62c admin admin] PATH_INFO: `/v3/projects/0d5d64912f024fe4aaff912f85bd10f4/users/ce7b46d961a3401db525c8913397f81b/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:00.772186 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c5395e-6082-4724-b304-599b00bfa62c admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d5d64912f024fe4aaff912f85bd10f4, user_id=ce7b46d961a3401db525c8913397f81b, role_id=5cf1f4bd2fef46ab9c6eff56d52f9f69)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:00.777066 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c5395e-6082-4724-b304-599b00bfa62c admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:00.845434 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 825/1651] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 12:01:00 2022] PUT /identity/v3/projects/0d5d64912f024fe4aaff912f85bd10f4/users/ce7b46d961a3401db525c8913397f81b/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:01:00.850504 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f630fc-2638-4499-9d57-e4bc61a5ab11 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:00.850830 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f630fc-2638-4499-9d57-e4bc61a5ab11 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:00.851111 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f630fc-2638-4499-9d57-e4bc61a5ab11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:00.952357 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-91f630fc-2638-4499-9d57-e4bc61a5ab11 None None] MFA Rules not processed for user `ce7b46d961a3401db525c8913397f81b`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:01:00.953757 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-91f630fc-2638-4499-9d57-e4bc61a5ab11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:01.124048 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 827/1652] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:01:00 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:01:01.204403 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c5395e-6082-4724-b304-599b00bfa62c admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:01.315173 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c5395e-6082-4724-b304-599b00bfa62c admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:01.326053 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0cc88f-e43c-4601-8014-4c66ba545506 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b1b651c73664d08accc6edf2f79bace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:01.326996 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0cc88f-e43c-4601-8014-4c66ba545506 service cinder] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:01.327440 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0cc88f-e43c-4601-8014-4c66ba545506 service cinder] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:01.327787 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0cc88f-e43c-4601-8014-4c66ba545506 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:01.341330 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0cc88f-e43c-4601-8014-4c66ba545506 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:01.343636 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0cc88f-e43c-4601-8014-4c66ba545506 service cinder] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:01.372688 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 826/1653] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:01:01 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:01:02.314256 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f630fc-2638-4499-9d57-e4bc61a5ab11 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:02.417130 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f630fc-2638-4499-9d57-e4bc61a5ab11 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:02.425450 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760cccb9-c996-4d6d-8aa9-3e4ce226df69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:02.426690 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760cccb9-c996-4d6d-8aa9-3e4ce226df69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:02.427186 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760cccb9-c996-4d6d-8aa9-3e4ce226df69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:02.427661 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760cccb9-c996-4d6d-8aa9-3e4ce226df69 admin admin] PATH_INFO: `/v3/users/fd5407c77f504c8aa832274eede4107a` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:02.429961 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760cccb9-c996-4d6d-8aa9-3e4ce226df69 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd5407c77f504c8aa832274eede4107a)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:02.433636 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760cccb9-c996-4d6d-8aa9-3e4ce226df69 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:02.503942 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:01:02.506254 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-760cccb9-c996-4d6d-8aa9-3e4ce226df69 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd5407c77f504c8aa832274eede4107a'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:02.529369 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-760cccb9-c996-4d6d-8aa9-3e4ce226df69 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd5407c77f504c8aa832274eede4107a'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:02.531008 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-760cccb9-c996-4d6d-8aa9-3e4ce226df69 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd5407c77f504c8aa832274eede4107a'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:02.544867 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-760cccb9-c996-4d6d-8aa9-3e4ce226df69 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd5407c77f504c8aa832274eede4107a'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:02.545889 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-760cccb9-c996-4d6d-8aa9-3e4ce226df69 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd5407c77f504c8aa832274eede4107a'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:02.565223 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 828/1654] 10.222.0.59 () {58 vars in 1224 bytes} [Tue Oct 11 12:01:02 2022] DELETE /identity/v3/users/fd5407c77f504c8aa832274eede4107a => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:01:02.570463 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2fb7b1-58cf-4eb2-9f46-4e2d589ad616 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:02.571426 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2fb7b1-58cf-4eb2-9f46-4e2d589ad616 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:02.571426 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2fb7b1-58cf-4eb2-9f46-4e2d589ad616 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:02.598687 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-1a2fb7b1-58cf-4eb2-9f46-4e2d589ad616 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:01:02.600004 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-1a2fb7b1-58cf-4eb2-9f46-4e2d589ad616 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:02.674788 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 827/1655] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:01:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:01:02.686183 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760cccb9-c996-4d6d-8aa9-3e4ce226df69 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:02.687659 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-760cccb9-c996-4d6d-8aa9-3e4ce226df69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:02.787901 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760cccb9-c996-4d6d-8aa9-3e4ce226df69 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:02.792564 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18f76dc-459c-4aae-9b94-825bebad1249 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e563fe174db4cc19ae6bf8162cc3d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:02.793284 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18f76dc-459c-4aae-9b94-825bebad1249 service neutron] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:02.793577 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18f76dc-459c-4aae-9b94-825bebad1249 service neutron] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:02.794136 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18f76dc-459c-4aae-9b94-825bebad1249 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:02.805621 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18f76dc-459c-4aae-9b94-825bebad1249 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:02.807198 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18f76dc-459c-4aae-9b94-825bebad1249 service neutron] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:02.829661 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 829/1656] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:01:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:01:03.231046 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2fb7b1-58cf-4eb2-9f46-4e2d589ad616 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:03.233572 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-1a2fb7b1-58cf-4eb2-9f46-4e2d589ad616 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:03.256836 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2fb7b1-58cf-4eb2-9f46-4e2d589ad616 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:03.265123 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000c14d2-81de-4438-8408-2eaeb7be5f0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:03.266220 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000c14d2-81de-4438-8408-2eaeb7be5f0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:03.266756 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000c14d2-81de-4438-8408-2eaeb7be5f0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:03.267227 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000c14d2-81de-4438-8408-2eaeb7be5f0c admin admin] PATH_INFO: `/v3/projects/caa7ebc594274b5c90fbd61e4b9f484a` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:03.269947 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000c14d2-81de-4438-8408-2eaeb7be5f0c admin admin] RBAC: Authorizing `identity:delete_project(project_id=caa7ebc594274b5c90fbd61e4b9f484a)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:03.274104 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000c14d2-81de-4438-8408-2eaeb7be5f0c admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:03.279559 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:01:03.352294 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-000c14d2-81de-4438-8408-2eaeb7be5f0c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'caa7ebc594274b5c90fbd61e4b9f484a', 'request_id': 'req-000c14d2-81de-4438-8408-2eaeb7be5f0c'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:03.359717 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-000c14d2-81de-4438-8408-2eaeb7be5f0c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'caa7ebc594274b5c90fbd61e4b9f484a', 'request_id': 'req-000c14d2-81de-4438-8408-2eaeb7be5f0c'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:03.370641 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-000c14d2-81de-4438-8408-2eaeb7be5f0c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'caa7ebc594274b5c90fbd61e4b9f484a', 'request_id': 'req-000c14d2-81de-4438-8408-2eaeb7be5f0c'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:03.389673 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-000c14d2-81de-4438-8408-2eaeb7be5f0c admin admin] The token cache is being invalidate because project caa7ebc594274b5c90fbd61e4b9f484a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 12:01:03.389974 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-000c14d2-81de-4438-8408-2eaeb7be5f0c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:03.390699 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-000c14d2-81de-4438-8408-2eaeb7be5f0c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:03.392382 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 828/1657] 10.222.0.59 () {58 vars in 1233 bytes} [Tue Oct 11 12:01:03 2022] DELETE /identity/v3/projects/caa7ebc594274b5c90fbd61e4b9f484a => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:01:03.397284 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18f76dc-459c-4aae-9b94-825bebad1249 service neutron] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:03.410114 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18f76dc-459c-4aae-9b94-825bebad1249 service neutron] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:03.503124 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdea607-3f42-44ab-8651-01818e988a3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:03.503895 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdea607-3f42-44ab-8651-01818e988a3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:03.504170 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdea607-3f42-44ab-8651-01818e988a3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:03.504494 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdea607-3f42-44ab-8651-01818e988a3b admin admin] PATH_INFO: `/v3/users/ce7b46d961a3401db525c8913397f81b` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:03.505886 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdea607-3f42-44ab-8651-01818e988a3b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce7b46d961a3401db525c8913397f81b)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:03.507875 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdea607-3f42-44ab-8651-01818e988a3b admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:03.568358 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-8bdea607-3f42-44ab-8651-01818e988a3b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce7b46d961a3401db525c8913397f81b'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:03.588503 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-8bdea607-3f42-44ab-8651-01818e988a3b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce7b46d961a3401db525c8913397f81b'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:03.589273 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-8bdea607-3f42-44ab-8651-01818e988a3b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce7b46d961a3401db525c8913397f81b'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:03.592506 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:01:03.599720 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-8bdea607-3f42-44ab-8651-01818e988a3b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce7b46d961a3401db525c8913397f81b'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:03.601063 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-8bdea607-3f42-44ab-8651-01818e988a3b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce7b46d961a3401db525c8913397f81b'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:03.620614 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 830/1658] 10.222.0.59 () {58 vars in 1224 bytes} [Tue Oct 11 12:01:03 2022] DELETE /identity/v3/users/ce7b46d961a3401db525c8913397f81b => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 12:01:04.259186 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000c14d2-81de-4438-8408-2eaeb7be5f0c admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:04.262605 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-000c14d2-81de-4438-8408-2eaeb7be5f0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:04.287783 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000c14d2-81de-4438-8408-2eaeb7be5f0c admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:04.375312 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5b54a3-6a94-49ca-a8d7-d059d3a22fdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:04.375984 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5b54a3-6a94-49ca-a8d7-d059d3a22fdf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:04.376262 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5b54a3-6a94-49ca-a8d7-d059d3a22fdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:04.376540 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5b54a3-6a94-49ca-a8d7-d059d3a22fdf admin admin] PATH_INFO: `/v3/projects/0d5d64912f024fe4aaff912f85bd10f4` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:04.377882 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5b54a3-6a94-49ca-a8d7-d059d3a22fdf admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d5d64912f024fe4aaff912f85bd10f4)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:04.379924 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5b54a3-6a94-49ca-a8d7-d059d3a22fdf admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:04.440779 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-bd5b54a3-6a94-49ca-a8d7-d059d3a22fdf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d5d64912f024fe4aaff912f85bd10f4', 'request_id': 'req-bd5b54a3-6a94-49ca-a8d7-d059d3a22fdf'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:04.450001 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-bd5b54a3-6a94-49ca-a8d7-d059d3a22fdf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d5d64912f024fe4aaff912f85bd10f4', 'request_id': 'req-bd5b54a3-6a94-49ca-a8d7-d059d3a22fdf'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:04.461261 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-bd5b54a3-6a94-49ca-a8d7-d059d3a22fdf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d5d64912f024fe4aaff912f85bd10f4', 'request_id': 'req-bd5b54a3-6a94-49ca-a8d7-d059d3a22fdf'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:04.476946 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-bd5b54a3-6a94-49ca-a8d7-d059d3a22fdf admin admin] The token cache is being invalidate because project 0d5d64912f024fe4aaff912f85bd10f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 12:01:04.477338 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-bd5b54a3-6a94-49ca-a8d7-d059d3a22fdf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:04.478143 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-bd5b54a3-6a94-49ca-a8d7-d059d3a22fdf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:04.479903 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 829/1659] 10.222.0.59 () {58 vars in 1233 bytes} [Tue Oct 11 12:01:04 2022] DELETE /identity/v3/projects/0d5d64912f024fe4aaff912f85bd10f4 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:01:04.486720 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f24fb40-548b-4cf4-ad32-f134e347dcbb None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:04.487449 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f24fb40-548b-4cf4-ad32-f134e347dcbb None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:04.487619 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f24fb40-548b-4cf4-ad32-f134e347dcbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:04.521247 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-0f24fb40-548b-4cf4-ad32-f134e347dcbb None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:01:04.523611 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-0f24fb40-548b-4cf4-ad32-f134e347dcbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:04.650707 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 831/1660] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:01:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:01:04.664044 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2f68eb-ef39-4b87-b90b-b5f0d714976c None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:04.664536 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2f68eb-ef39-4b87-b90b-b5f0d714976c None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:04.664999 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2f68eb-ef39-4b87-b90b-b5f0d714976c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:04.700056 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-de2f68eb-ef39-4b87-b90b-b5f0d714976c None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:01:04.702332 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-de2f68eb-ef39-4b87-b90b-b5f0d714976c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:04.735158 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 830/1661] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:01:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:01:04.740638 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f24fb40-548b-4cf4-ad32-f134e347dcbb None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:04.760001 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f24fb40-548b-4cf4-ad32-f134e347dcbb None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:04.788207 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e09588-52ed-44f4-b07e-85694631e110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:04.789281 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e09588-52ed-44f4-b07e-85694631e110 admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:04.789812 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e09588-52ed-44f4-b07e-85694631e110 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:04.790483 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e09588-52ed-44f4-b07e-85694631e110 admin admin] PATH_INFO: `/v3/domains` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:04.791733 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e09588-52ed-44f4-b07e-85694631e110 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 12:01:04.792263 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e09588-52ed-44f4-b07e-85694631e110 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:04.797126 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e09588-52ed-44f4-b07e-85694631e110 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:04.828190 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 832/1662] 10.222.0.59 () {58 vars in 1153 bytes} [Tue Oct 11 12:01:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 12:01:04.830314 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2f68eb-ef39-4b87-b90b-b5f0d714976c None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:04.861580 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2f68eb-ef39-4b87-b90b-b5f0d714976c None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:04.870745 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f504b76a-396a-4a35-951c-3faf25a6e09b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:04.871988 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f504b76a-396a-4a35-951c-3faf25a6e09b admin admin] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:04.872941 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f504b76a-396a-4a35-951c-3faf25a6e09b admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:04.873295 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f504b76a-396a-4a35-951c-3faf25a6e09b admin admin] PATH_INFO: `/v3/projects` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:04.874676 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f504b76a-396a-4a35-951c-3faf25a6e09b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:04.878012 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f504b76a-396a-4a35-951c-3faf25a6e09b admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:04.914966 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:01:04.976954 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 831/1663] 10.222.0.59 () {60 vars in 1153 bytes} [Tue Oct 11 12:01:04 2022] POST /identity/v3/projects => generated 369 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:01:04.977769 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e09588-52ed-44f4-b07e-85694631e110 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:04.993458 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e09588-52ed-44f4-b07e-85694631e110 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:05.163098 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abf470c-3c54-4dbf-b97b-7f6c218c665a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:05.163602 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abf470c-3c54-4dbf-b97b-7f6c218c665a admin admin] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:05.164171 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abf470c-3c54-4dbf-b97b-7f6c218c665a admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:05.164521 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abf470c-3c54-4dbf-b97b-7f6c218c665a admin admin] PATH_INFO: `/v3/users` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:05.165424 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abf470c-3c54-4dbf-b97b-7f6c218c665a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:05.167675 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abf470c-3c54-4dbf-b97b-7f6c218c665a admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:05.224162 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 833/1664] 10.222.0.59 () {60 vars in 1144 bytes} [Tue Oct 11 12:01:04 2022] POST /identity/v3/users => generated 285 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 12:01:05.230731 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f504b76a-396a-4a35-951c-3faf25a6e09b admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:05.257902 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f504b76a-396a-4a35-951c-3faf25a6e09b admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:05.265368 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a07580-2887-4b9b-b39b-63a701754ffb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:05.266786 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a07580-2887-4b9b-b39b-63a701754ffb admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:05.267317 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a07580-2887-4b9b-b39b-63a701754ffb admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:05.267749 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a07580-2887-4b9b-b39b-63a701754ffb admin admin] PATH_INFO: `/v3/roles` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:05.268612 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a07580-2887-4b9b-b39b-63a701754ffb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:05.271249 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a07580-2887-4b9b-b39b-63a701754ffb admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:05.287255 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 832/1665] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:01:05 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:01:05.292791 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abf470c-3c54-4dbf-b97b-7f6c218c665a admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:05.313758 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abf470c-3c54-4dbf-b97b-7f6c218c665a admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:05.320282 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec81946-4241-4243-83a8-33b699ba9d58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:05.320955 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec81946-4241-4243-83a8-33b699ba9d58 admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:05.321281 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec81946-4241-4243-83a8-33b699ba9d58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:05.321630 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec81946-4241-4243-83a8-33b699ba9d58 admin admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:05.323152 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec81946-4241-4243-83a8-33b699ba9d58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:05.325025 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec81946-4241-4243-83a8-33b699ba9d58 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:05.362009 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a07580-2887-4b9b-b39b-63a701754ffb admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:05.367142 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 834/1666] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:01:05 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:01:05.381084 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a07580-2887-4b9b-b39b-63a701754ffb admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:05.405535 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d416a558-8431-4f74-b684-e0d3ce9034a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:05.407111 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d416a558-8431-4f74-b684-e0d3ce9034a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:05.407474 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d416a558-8431-4f74-b684-e0d3ce9034a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:05.408125 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d416a558-8431-4f74-b684-e0d3ce9034a2 admin admin] PATH_INFO: `/v3/projects/9fcb546ffbbe429fb588c7ae2bdddac8/users/5c8d10f2138e482b94b954d73b5942c4/roles/de2d2bd26fff4748a79990f97a107164` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:05.449787 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d416a558-8431-4f74-b684-e0d3ce9034a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9fcb546ffbbe429fb588c7ae2bdddac8, user_id=5c8d10f2138e482b94b954d73b5942c4, role_id=de2d2bd26fff4748a79990f97a107164)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:05.458194 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d416a558-8431-4f74-b684-e0d3ce9034a2 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:05.490481 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 833/1667] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 12:01:05 2022] PUT /identity/v3/projects/9fcb546ffbbe429fb588c7ae2bdddac8/users/5c8d10f2138e482b94b954d73b5942c4/roles/de2d2bd26fff4748a79990f97a107164 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:01:05.511771 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842fc92a-1f11-469e-95e5-f80b25032ec0 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:05.513395 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842fc92a-1f11-469e-95e5-f80b25032ec0 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:05.513896 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842fc92a-1f11-469e-95e5-f80b25032ec0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:05.577472 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-842fc92a-1f11-469e-95e5-f80b25032ec0 None None] MFA Rules not processed for user `5c8d10f2138e482b94b954d73b5942c4`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:01:05.579024 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-842fc92a-1f11-469e-95e5-f80b25032ec0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:05.654998 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:01:05.714895 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 835/1668] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:01:05 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:01:05.719775 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d416a558-8431-4f74-b684-e0d3ce9034a2 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:05.853330 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d416a558-8431-4f74-b684-e0d3ce9034a2 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:05.867836 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c27a3cb-8dc9-48b4-8204-148474cfc626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:05.869134 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c27a3cb-8dc9-48b4-8204-148474cfc626 admin admin] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:05.869533 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c27a3cb-8dc9-48b4-8204-148474cfc626 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:05.870445 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c27a3cb-8dc9-48b4-8204-148474cfc626 admin admin] PATH_INFO: `/v3/projects` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:05.872344 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c27a3cb-8dc9-48b4-8204-148474cfc626 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:05.878939 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c27a3cb-8dc9-48b4-8204-148474cfc626 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:05.929535 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 834/1669] 10.222.0.59 () {60 vars in 1153 bytes} [Tue Oct 11 12:01:05 2022] POST /identity/v3/projects => generated 369 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:01:05.934494 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842fc92a-1f11-469e-95e5-f80b25032ec0 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:05.953719 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842fc92a-1f11-469e-95e5-f80b25032ec0 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:06.091292 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d06389-38a7-425b-af6b-a071f69f7318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:06.091947 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d06389-38a7-425b-af6b-a071f69f7318 admin admin] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:06.092223 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d06389-38a7-425b-af6b-a071f69f7318 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:06.092499 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d06389-38a7-425b-af6b-a071f69f7318 admin admin] PATH_INFO: `/v3/users` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:06.093305 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d06389-38a7-425b-af6b-a071f69f7318 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:06.095427 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d06389-38a7-425b-af6b-a071f69f7318 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:06.127482 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 836/1670] 10.222.0.59 () {60 vars in 1144 bytes} [Tue Oct 11 12:01:05 2022] POST /identity/v3/users => generated 291 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:01:06.133581 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c27a3cb-8dc9-48b4-8204-148474cfc626 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:06.153613 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c27a3cb-8dc9-48b4-8204-148474cfc626 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:06.168722 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dd4966-02ea-439c-a23e-6faa816f9ed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:06.170180 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dd4966-02ea-439c-a23e-6faa816f9ed6 admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:06.171019 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dd4966-02ea-439c-a23e-6faa816f9ed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:06.171602 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dd4966-02ea-439c-a23e-6faa816f9ed6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:06.172982 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26dd4966-02ea-439c-a23e-6faa816f9ed6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:06.176566 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26dd4966-02ea-439c-a23e-6faa816f9ed6 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:06.210168 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 835/1671] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:01:06 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 11 12:01:06.212057 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d06389-38a7-425b-af6b-a071f69f7318 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:06.223443 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d06389-38a7-425b-af6b-a071f69f7318 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:06.228648 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a173ff-27e0-49ff-8ddd-7b18c04d11b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:06.229510 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a173ff-27e0-49ff-8ddd-7b18c04d11b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:06.229798 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a173ff-27e0-49ff-8ddd-7b18c04d11b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:06.230106 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a173ff-27e0-49ff-8ddd-7b18c04d11b4 admin admin] PATH_INFO: `/v3/projects/96244b408c054f64bdfd02a5c994ace3/users/6cf3c328623c48e8bf1baa5384a4e465/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:06.248422 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a173ff-27e0-49ff-8ddd-7b18c04d11b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=96244b408c054f64bdfd02a5c994ace3, user_id=6cf3c328623c48e8bf1baa5384a4e465, role_id=5cf1f4bd2fef46ab9c6eff56d52f9f69)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:06.251500 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a173ff-27e0-49ff-8ddd-7b18c04d11b4 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:06.268817 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 837/1672] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 12:01:06 2022] PUT /identity/v3/projects/96244b408c054f64bdfd02a5c994ace3/users/6cf3c328623c48e8bf1baa5384a4e465/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 12:01:06.273428 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dd4966-02ea-439c-a23e-6faa816f9ed6 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:06.286497 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dd4966-02ea-439c-a23e-6faa816f9ed6 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:06.366981 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69f1f07-f04c-42ac-a1bc-e6ed9476c62a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:06.367919 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69f1f07-f04c-42ac-a1bc-e6ed9476c62a admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:06.368252 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69f1f07-f04c-42ac-a1bc-e6ed9476c62a admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:06.368616 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69f1f07-f04c-42ac-a1bc-e6ed9476c62a admin admin] PATH_INFO: `/v3/roles` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:06.369485 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69f1f07-f04c-42ac-a1bc-e6ed9476c62a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:06.371263 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69f1f07-f04c-42ac-a1bc-e6ed9476c62a admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:06.384104 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 836/1673] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:01:06 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:01:06.389519 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a173ff-27e0-49ff-8ddd-7b18c04d11b4 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:06.402532 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a173ff-27e0-49ff-8ddd-7b18c04d11b4 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:06.409615 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd9a19b-91fa-49d4-bde3-55353d824ea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:06.410400 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd9a19b-91fa-49d4-bde3-55353d824ea2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:06.410860 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd9a19b-91fa-49d4-bde3-55353d824ea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:06.411157 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd9a19b-91fa-49d4-bde3-55353d824ea2 admin admin] PATH_INFO: `/v3/projects/96244b408c054f64bdfd02a5c994ace3/users/6cf3c328623c48e8bf1baa5384a4e465/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:06.413201 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd9a19b-91fa-49d4-bde3-55353d824ea2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=96244b408c054f64bdfd02a5c994ace3, user_id=6cf3c328623c48e8bf1baa5384a4e465, role_id=5cf1f4bd2fef46ab9c6eff56d52f9f69)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:06.416092 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd9a19b-91fa-49d4-bde3-55353d824ea2 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:06.433739 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 838/1674] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 12:01:06 2022] PUT /identity/v3/projects/96244b408c054f64bdfd02a5c994ace3/users/6cf3c328623c48e8bf1baa5384a4e465/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:01:06.452850 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcd067e-a749-4ad1-b246-030bd1bdaf8d None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:06.453366 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcd067e-a749-4ad1-b246-030bd1bdaf8d None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:06.453844 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcd067e-a749-4ad1-b246-030bd1bdaf8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:06.508216 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-0fcd067e-a749-4ad1-b246-030bd1bdaf8d None None] MFA Rules not processed for user `6cf3c328623c48e8bf1baa5384a4e465`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:01:06.510092 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-0fcd067e-a749-4ad1-b246-030bd1bdaf8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:06.546434 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:01:06.672332 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 837/1675] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:01:06 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:01:06.706318 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd9a19b-91fa-49d4-bde3-55353d824ea2 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:06.708895 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-9bd9a19b-91fa-49d4-bde3-55353d824ea2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:06.924919 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd9a19b-91fa-49d4-bde3-55353d824ea2 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:06.935138 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4234f338-d357-44ba-b398-7e70ce0dc6c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b1b651c73664d08accc6edf2f79bace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:06.937357 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4234f338-d357-44ba-b398-7e70ce0dc6c8 service cinder] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:06.938106 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4234f338-d357-44ba-b398-7e70ce0dc6c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:06.938903 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4234f338-d357-44ba-b398-7e70ce0dc6c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:06.962057 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4234f338-d357-44ba-b398-7e70ce0dc6c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:06.965106 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4234f338-d357-44ba-b398-7e70ce0dc6c8 service cinder] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:06.995840 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 839/1676] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:01:06 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:01:08.414281 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcd067e-a749-4ad1-b246-030bd1bdaf8d None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:08.415986 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-0fcd067e-a749-4ad1-b246-030bd1bdaf8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:08.551653 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcd067e-a749-4ad1-b246-030bd1bdaf8d None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:08.557179 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce2235c-0a9b-4acd-a326-0c86f23d0dec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:08.558120 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce2235c-0a9b-4acd-a326-0c86f23d0dec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:08.558408 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce2235c-0a9b-4acd-a326-0c86f23d0dec admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:08.558709 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce2235c-0a9b-4acd-a326-0c86f23d0dec admin admin] PATH_INFO: `/v3/users/df4f56462f6d44d49a0cb547915d8df5` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:08.559959 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce2235c-0a9b-4acd-a326-0c86f23d0dec admin admin] RBAC: Authorizing `identity:delete_user(user_id=df4f56462f6d44d49a0cb547915d8df5)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:08.562086 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce2235c-0a9b-4acd-a326-0c86f23d0dec admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:08.659780 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-0ce2235c-0a9b-4acd-a326-0c86f23d0dec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df4f56462f6d44d49a0cb547915d8df5'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:08.711022 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-0ce2235c-0a9b-4acd-a326-0c86f23d0dec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df4f56462f6d44d49a0cb547915d8df5'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:08.755095 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-0ce2235c-0a9b-4acd-a326-0c86f23d0dec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df4f56462f6d44d49a0cb547915d8df5'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:08.755095 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-0ce2235c-0a9b-4acd-a326-0c86f23d0dec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df4f56462f6d44d49a0cb547915d8df5'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:08.764490 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-0ce2235c-0a9b-4acd-a326-0c86f23d0dec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df4f56462f6d44d49a0cb547915d8df5'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:08.777394 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 838/1677] 10.222.0.59 () {58 vars in 1224 bytes} [Tue Oct 11 12:01:08 2022] DELETE /identity/v3/users/df4f56462f6d44d49a0cb547915d8df5 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:01:08.786188 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afde9127-76e2-4604-9f3a-6fcdb28d1c44 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:08.786923 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afde9127-76e2-4604-9f3a-6fcdb28d1c44 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:08.786923 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afde9127-76e2-4604-9f3a-6fcdb28d1c44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:08.839972 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-afde9127-76e2-4604-9f3a-6fcdb28d1c44 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:01:08.842984 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-afde9127-76e2-4604-9f3a-6fcdb28d1c44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:08.873710 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce2235c-0a9b-4acd-a326-0c86f23d0dec admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:08.875559 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-0ce2235c-0a9b-4acd-a326-0c86f23d0dec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:08.945598 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:01:08.974698 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:01:09.023784 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 840/1678] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:01:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:01:09.035362 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce2235c-0a9b-4acd-a326-0c86f23d0dec admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:09.038328 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afde9127-76e2-4604-9f3a-6fcdb28d1c44 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:09.039863 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-afde9127-76e2-4604-9f3a-6fcdb28d1c44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:09.058856 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afb901e-f3d3-4724-8b79-fde37912f3f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:09.059568 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afb901e-f3d3-4724-8b79-fde37912f3f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:09.060892 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afb901e-f3d3-4724-8b79-fde37912f3f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:09.061216 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afb901e-f3d3-4724-8b79-fde37912f3f4 admin admin] PATH_INFO: `/v3/users/9ef6c9ae145e4a3f898d50cef8f3c0e3` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:09.063861 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afb901e-f3d3-4724-8b79-fde37912f3f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ef6c9ae145e4a3f898d50cef8f3c0e3)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:09.066023 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afb901e-f3d3-4724-8b79-fde37912f3f4 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:09.166010 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-3afb901e-f3d3-4724-8b79-fde37912f3f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ef6c9ae145e4a3f898d50cef8f3c0e3'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:09.201064 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-3afb901e-f3d3-4724-8b79-fde37912f3f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ef6c9ae145e4a3f898d50cef8f3c0e3'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:09.206534 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afde9127-76e2-4604-9f3a-6fcdb28d1c44 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:09.228771 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac1bcf9-462b-48a1-a2c4-1b074730fece service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e563fe174db4cc19ae6bf8162cc3d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:09.229145 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-3afb901e-f3d3-4724-8b79-fde37912f3f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ef6c9ae145e4a3f898d50cef8f3c0e3'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:09.229843 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-3afb901e-f3d3-4724-8b79-fde37912f3f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ef6c9ae145e4a3f898d50cef8f3c0e3'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:09.230473 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac1bcf9-462b-48a1-a2c4-1b074730fece service neutron] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:09.231953 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac1bcf9-462b-48a1-a2c4-1b074730fece service neutron] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:09.232242 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac1bcf9-462b-48a1-a2c4-1b074730fece service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:09.245748 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-3afb901e-f3d3-4724-8b79-fde37912f3f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ef6c9ae145e4a3f898d50cef8f3c0e3'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:09.263193 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 839/1679] 10.222.0.59 () {58 vars in 1224 bytes} [Tue Oct 11 12:01:08 2022] DELETE /identity/v3/users/9ef6c9ae145e4a3f898d50cef8f3c0e3 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:01:09.264873 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afb901e-f3d3-4724-8b79-fde37912f3f4 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:09.271168 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-3afb901e-f3d3-4724-8b79-fde37912f3f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:09.271637 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac1bcf9-462b-48a1-a2c4-1b074730fece service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:09.273663 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac1bcf9-462b-48a1-a2c4-1b074730fece service neutron] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:09.343255 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 841/1680] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:01:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:01:09.535280 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afb901e-f3d3-4724-8b79-fde37912f3f4 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:09.550601 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fc336c-1152-42c1-a584-6185b36e3a85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:09.550601 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fc336c-1152-42c1-a584-6185b36e3a85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:09.551257 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fc336c-1152-42c1-a584-6185b36e3a85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:09.551257 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fc336c-1152-42c1-a584-6185b36e3a85 admin admin] PATH_INFO: `/v3/users/8e8ff3ba800643b9bf478efdd101004b` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:09.553623 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fc336c-1152-42c1-a584-6185b36e3a85 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e8ff3ba800643b9bf478efdd101004b)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:09.558575 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fc336c-1152-42c1-a584-6185b36e3a85 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:09.793646 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-10fc336c-1152-42c1-a584-6185b36e3a85 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e8ff3ba800643b9bf478efdd101004b'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:09.807847 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:01:09.837612 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-10fc336c-1152-42c1-a584-6185b36e3a85 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e8ff3ba800643b9bf478efdd101004b'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:09.860027 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-10fc336c-1152-42c1-a584-6185b36e3a85 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e8ff3ba800643b9bf478efdd101004b'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:09.861592 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-10fc336c-1152-42c1-a584-6185b36e3a85 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e8ff3ba800643b9bf478efdd101004b'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:09.891055 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-10fc336c-1152-42c1-a584-6185b36e3a85 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e8ff3ba800643b9bf478efdd101004b'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:09.902492 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 840/1681] 10.222.0.59 () {58 vars in 1224 bytes} [Tue Oct 11 12:01:09 2022] DELETE /identity/v3/users/8e8ff3ba800643b9bf478efdd101004b => generated 0 bytes in 639 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:01:09.909344 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f4498c-6f9b-47c7-aeb6-2e4e5f9d4fe2 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:09.909942 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f4498c-6f9b-47c7-aeb6-2e4e5f9d4fe2 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:09.909942 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f4498c-6f9b-47c7-aeb6-2e4e5f9d4fe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:09.945890 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-72f4498c-6f9b-47c7-aeb6-2e4e5f9d4fe2 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:01:09.951407 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-72f4498c-6f9b-47c7-aeb6-2e4e5f9d4fe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:10.049700 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fc336c-1152-42c1-a584-6185b36e3a85 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:10.052194 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-10fc336c-1152-42c1-a584-6185b36e3a85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:10.131090 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 842/1682] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:01:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:01:10.187023 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f4498c-6f9b-47c7-aeb6-2e4e5f9d4fe2 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:10.188629 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-72f4498c-6f9b-47c7-aeb6-2e4e5f9d4fe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:10.287881 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fc336c-1152-42c1-a584-6185b36e3a85 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:10.298398 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e0e6c0-da69-461d-a583-e21163c1486a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:10.299150 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e0e6c0-da69-461d-a583-e21163c1486a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:10.299564 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e0e6c0-da69-461d-a583-e21163c1486a admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:10.300040 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e0e6c0-da69-461d-a583-e21163c1486a admin admin] PATH_INFO: `/v3/projects/4a8ea24a20dc486c9f504488c46fef97` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:10.302271 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e0e6c0-da69-461d-a583-e21163c1486a admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a8ea24a20dc486c9f504488c46fef97)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:10.304253 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e0e6c0-da69-461d-a583-e21163c1486a admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:10.398976 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-b2e0e6c0-da69-461d-a583-e21163c1486a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a8ea24a20dc486c9f504488c46fef97', 'request_id': 'req-b2e0e6c0-da69-461d-a583-e21163c1486a'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:10.433404 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f4498c-6f9b-47c7-aeb6-2e4e5f9d4fe2 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:10.438111 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-b2e0e6c0-da69-461d-a583-e21163c1486a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a8ea24a20dc486c9f504488c46fef97', 'request_id': 'req-b2e0e6c0-da69-461d-a583-e21163c1486a'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:10.451634 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-b2e0e6c0-da69-461d-a583-e21163c1486a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a8ea24a20dc486c9f504488c46fef97', 'request_id': 'req-b2e0e6c0-da69-461d-a583-e21163c1486a'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:10.505394 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-b2e0e6c0-da69-461d-a583-e21163c1486a admin admin] The token cache is being invalidate because project 4a8ea24a20dc486c9f504488c46fef97 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 12:01:10.506182 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-b2e0e6c0-da69-461d-a583-e21163c1486a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:10.515290 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-b2e0e6c0-da69-461d-a583-e21163c1486a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:10.522196 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 841/1683] 10.222.0.59 () {58 vars in 1233 bytes} [Tue Oct 11 12:01:10 2022] DELETE /identity/v3/projects/4a8ea24a20dc486c9f504488c46fef97 => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:01:10.522196 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e0e6c0-da69-461d-a583-e21163c1486a admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:10.524809 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-b2e0e6c0-da69-461d-a583-e21163c1486a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:10.555088 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:01:10.606880 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f562e41c-2dc9-41ad-a2bf-7e2c53c7885f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e563fe174db4cc19ae6bf8162cc3d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:10.607932 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f562e41c-2dc9-41ad-a2bf-7e2c53c7885f service neutron] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:10.608375 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f562e41c-2dc9-41ad-a2bf-7e2c53c7885f service neutron] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:10.608819 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f562e41c-2dc9-41ad-a2bf-7e2c53c7885f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:10.616406 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-f562e41c-2dc9-41ad-a2bf-7e2c53c7885f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:10.737601 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e0e6c0-da69-461d-a583-e21163c1486a admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:10.765113 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee22568d-2aea-4d28-8001-10635b12e846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:10.767975 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee22568d-2aea-4d28-8001-10635b12e846 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:10.768436 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee22568d-2aea-4d28-8001-10635b12e846 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:10.768908 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee22568d-2aea-4d28-8001-10635b12e846 admin admin] PATH_INFO: `/v3/projects/1a802749244d43aeb29dfb2e029d2793` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:10.773350 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee22568d-2aea-4d28-8001-10635b12e846 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a802749244d43aeb29dfb2e029d2793)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:10.783158 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee22568d-2aea-4d28-8001-10635b12e846 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:10.789876 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:01:10.797372 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f562e41c-2dc9-41ad-a2bf-7e2c53c7885f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:10.809789 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f562e41c-2dc9-41ad-a2bf-7e2c53c7885f service neutron] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:10.842532 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 843/1684] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:01:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 656 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 11 12:01:10.844792 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f562e41c-2dc9-41ad-a2bf-7e2c53c7885f service neutron] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:10.848100 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-f562e41c-2dc9-41ad-a2bf-7e2c53c7885f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:10.886978 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f562e41c-2dc9-41ad-a2bf-7e2c53c7885f service neutron] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:10.913052 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0230591-accd-4a61-89e4-7e0334d18fa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:10.915124 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0230591-accd-4a61-89e4-7e0334d18fa4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:10.915554 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0230591-accd-4a61-89e4-7e0334d18fa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:10.916017 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0230591-accd-4a61-89e4-7e0334d18fa4 admin admin] PATH_INFO: `/v3/users/b0be132a1e3c4ba882c61b1e19de35ac` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:10.921141 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0230591-accd-4a61-89e4-7e0334d18fa4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0be132a1e3c4ba882c61b1e19de35ac)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:10.927495 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0230591-accd-4a61-89e4-7e0334d18fa4 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:10.953388 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-ee22568d-2aea-4d28-8001-10635b12e846 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a802749244d43aeb29dfb2e029d2793', 'request_id': 'req-ee22568d-2aea-4d28-8001-10635b12e846'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:10.981239 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-ee22568d-2aea-4d28-8001-10635b12e846 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a802749244d43aeb29dfb2e029d2793', 'request_id': 'req-ee22568d-2aea-4d28-8001-10635b12e846'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:11.005077 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-ee22568d-2aea-4d28-8001-10635b12e846 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a802749244d43aeb29dfb2e029d2793', 'request_id': 'req-ee22568d-2aea-4d28-8001-10635b12e846'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:11.023826 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-a0230591-accd-4a61-89e4-7e0334d18fa4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0be132a1e3c4ba882c61b1e19de35ac'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:11.043682 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-a0230591-accd-4a61-89e4-7e0334d18fa4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0be132a1e3c4ba882c61b1e19de35ac'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:11.045000 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-a0230591-accd-4a61-89e4-7e0334d18fa4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0be132a1e3c4ba882c61b1e19de35ac'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:11.057430 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-a0230591-accd-4a61-89e4-7e0334d18fa4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0be132a1e3c4ba882c61b1e19de35ac'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:11.058321 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-a0230591-accd-4a61-89e4-7e0334d18fa4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0be132a1e3c4ba882c61b1e19de35ac'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:11.059681 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-ee22568d-2aea-4d28-8001-10635b12e846 admin admin] The token cache is being invalidate because project 1a802749244d43aeb29dfb2e029d2793 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 12:01:11.060317 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-ee22568d-2aea-4d28-8001-10635b12e846 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:11.061528 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-ee22568d-2aea-4d28-8001-10635b12e846 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:11.064686 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 842/1685] 10.222.0.59 () {58 vars in 1233 bytes} [Tue Oct 11 12:01:10 2022] DELETE /identity/v3/projects/1a802749244d43aeb29dfb2e029d2793 => generated 0 bytes in 546 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:01:11.066331 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee22568d-2aea-4d28-8001-10635b12e846 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:11.069313 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-ee22568d-2aea-4d28-8001-10635b12e846 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:11.085576 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 844/1686] 10.222.0.59 () {58 vars in 1224 bytes} [Tue Oct 11 12:01:10 2022] DELETE /identity/v3/users/b0be132a1e3c4ba882c61b1e19de35ac => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:01:11.087219 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0230591-accd-4a61-89e4-7e0334d18fa4 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:11.089225 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-a0230591-accd-4a61-89e4-7e0334d18fa4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:11.196746 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:01:11.202056 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0230591-accd-4a61-89e4-7e0334d18fa4 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:11.212958 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a236d53-1018-405c-b558-b8b31c8e96dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:11.213760 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a236d53-1018-405c-b558-b8b31c8e96dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:11.214436 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a236d53-1018-405c-b558-b8b31c8e96dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:11.214977 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a236d53-1018-405c-b558-b8b31c8e96dc admin admin] PATH_INFO: `/v3/users/8861e549cda14ce89b0dbe7f8e5bf808` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:11.216684 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a236d53-1018-405c-b558-b8b31c8e96dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=8861e549cda14ce89b0dbe7f8e5bf808)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:11.219629 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a236d53-1018-405c-b558-b8b31c8e96dc admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:11.234369 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee22568d-2aea-4d28-8001-10635b12e846 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:11.251886 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1b0f12-a9fa-48a1-9bdf-45fae1fbb6cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:11.253602 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1b0f12-a9fa-48a1-9bdf-45fae1fbb6cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:11.254328 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1b0f12-a9fa-48a1-9bdf-45fae1fbb6cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:11.254972 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1b0f12-a9fa-48a1-9bdf-45fae1fbb6cc admin admin] PATH_INFO: `/v3/users/5c8d10f2138e482b94b954d73b5942c4` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:11.260344 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1b0f12-a9fa-48a1-9bdf-45fae1fbb6cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c8d10f2138e482b94b954d73b5942c4)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:11.267192 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1b0f12-a9fa-48a1-9bdf-45fae1fbb6cc admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:11.293925 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-4a236d53-1018-405c-b558-b8b31c8e96dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8861e549cda14ce89b0dbe7f8e5bf808'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:11.318679 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-4a236d53-1018-405c-b558-b8b31c8e96dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8861e549cda14ce89b0dbe7f8e5bf808'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:11.320114 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-4a236d53-1018-405c-b558-b8b31c8e96dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8861e549cda14ce89b0dbe7f8e5bf808'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:11.334120 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-4a236d53-1018-405c-b558-b8b31c8e96dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8861e549cda14ce89b0dbe7f8e5bf808'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:11.337377 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-4a236d53-1018-405c-b558-b8b31c8e96dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8861e549cda14ce89b0dbe7f8e5bf808'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:11.357408 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 845/1687] 10.222.0.59 () {58 vars in 1224 bytes} [Tue Oct 11 12:01:11 2022] DELETE /identity/v3/users/8861e549cda14ce89b0dbe7f8e5bf808 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:01:11.381292 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-8c1b0f12-a9fa-48a1-9bdf-45fae1fbb6cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c8d10f2138e482b94b954d73b5942c4'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:11.419490 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-8c1b0f12-a9fa-48a1-9bdf-45fae1fbb6cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c8d10f2138e482b94b954d73b5942c4'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:11.460447 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-8c1b0f12-a9fa-48a1-9bdf-45fae1fbb6cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c8d10f2138e482b94b954d73b5942c4'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:11.462339 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-8c1b0f12-a9fa-48a1-9bdf-45fae1fbb6cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c8d10f2138e482b94b954d73b5942c4'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:11.480622 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-8c1b0f12-a9fa-48a1-9bdf-45fae1fbb6cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c8d10f2138e482b94b954d73b5942c4'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:11.499158 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 843/1688] 10.222.0.59 () {58 vars in 1224 bytes} [Tue Oct 11 12:01:11 2022] DELETE /identity/v3/users/5c8d10f2138e482b94b954d73b5942c4 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:01:11.500657 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f467167-faa9-4dbb-a255-9e30712b3177 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:11.500949 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f467167-faa9-4dbb-a255-9e30712b3177 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:11.501227 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f467167-faa9-4dbb-a255-9e30712b3177 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:11.535257 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-2f467167-faa9-4dbb-a255-9e30712b3177 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:01:11.537305 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-2f467167-faa9-4dbb-a255-9e30712b3177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:11.653495 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1b0f12-a9fa-48a1-9bdf-45fae1fbb6cc admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:11.656762 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-8c1b0f12-a9fa-48a1-9bdf-45fae1fbb6cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:11.680113 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 846/1689] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:01:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:01:11.697776 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f467167-faa9-4dbb-a255-9e30712b3177 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:11.700166 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-2f467167-faa9-4dbb-a255-9e30712b3177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:11.727011 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1b0f12-a9fa-48a1-9bdf-45fae1fbb6cc admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:11.745631 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaf1b77-bcf3-4b0f-89bd-86aaeef8aff5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:11.746712 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaf1b77-bcf3-4b0f-89bd-86aaeef8aff5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:11.747234 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaf1b77-bcf3-4b0f-89bd-86aaeef8aff5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:11.748649 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaf1b77-bcf3-4b0f-89bd-86aaeef8aff5 admin admin] PATH_INFO: `/v3/projects/c7834515e94848c38d4b619514eb7cfc` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:11.750310 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abaf1b77-bcf3-4b0f-89bd-86aaeef8aff5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7834515e94848c38d4b619514eb7cfc)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:11.752722 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abaf1b77-bcf3-4b0f-89bd-86aaeef8aff5 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:11.767950 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:01:11.840803 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f467167-faa9-4dbb-a255-9e30712b3177 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:11.851604 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-abaf1b77-bcf3-4b0f-89bd-86aaeef8aff5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7834515e94848c38d4b619514eb7cfc', 'request_id': 'req-abaf1b77-bcf3-4b0f-89bd-86aaeef8aff5'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:11.859831 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-abaf1b77-bcf3-4b0f-89bd-86aaeef8aff5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7834515e94848c38d4b619514eb7cfc', 'request_id': 'req-abaf1b77-bcf3-4b0f-89bd-86aaeef8aff5'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:11.872612 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-abaf1b77-bcf3-4b0f-89bd-86aaeef8aff5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7834515e94848c38d4b619514eb7cfc', 'request_id': 'req-abaf1b77-bcf3-4b0f-89bd-86aaeef8aff5'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:11.885769 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:01:11.907516 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-abaf1b77-bcf3-4b0f-89bd-86aaeef8aff5 admin admin] The token cache is being invalidate because project c7834515e94848c38d4b619514eb7cfc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 12:01:11.908770 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-abaf1b77-bcf3-4b0f-89bd-86aaeef8aff5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:11.910132 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-abaf1b77-bcf3-4b0f-89bd-86aaeef8aff5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:11.918934 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 844/1690] 10.222.0.59 () {58 vars in 1233 bytes} [Tue Oct 11 12:01:11 2022] DELETE /identity/v3/projects/c7834515e94848c38d4b619514eb7cfc => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:01:11.920711 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaf1b77-bcf3-4b0f-89bd-86aaeef8aff5 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:11.924449 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-abaf1b77-bcf3-4b0f-89bd-86aaeef8aff5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:11.942320 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34b3efe-ef6d-46c7-ad5c-487d1e67cc38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e563fe174db4cc19ae6bf8162cc3d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:11.943760 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34b3efe-ef6d-46c7-ad5c-487d1e67cc38 service neutron] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:11.944387 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34b3efe-ef6d-46c7-ad5c-487d1e67cc38 service neutron] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:11.944605 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34b3efe-ef6d-46c7-ad5c-487d1e67cc38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:11.949338 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-a34b3efe-ef6d-46c7-ad5c-487d1e67cc38 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:12.103416 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34b3efe-ef6d-46c7-ad5c-487d1e67cc38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:12.104961 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34b3efe-ef6d-46c7-ad5c-487d1e67cc38 service neutron] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:12.112308 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaf1b77-bcf3-4b0f-89bd-86aaeef8aff5 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:12.124044 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a031d0b-8edc-4182-996c-ab1dd114c1b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:12.125293 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a031d0b-8edc-4182-996c-ab1dd114c1b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:12.125293 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a031d0b-8edc-4182-996c-ab1dd114c1b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:12.125480 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a031d0b-8edc-4182-996c-ab1dd114c1b5 admin admin] PATH_INFO: `/v3/projects/67c854dbc3024ce28885a7eb854c8de4` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:12.128048 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a031d0b-8edc-4182-996c-ab1dd114c1b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=67c854dbc3024ce28885a7eb854c8de4)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:12.130533 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a031d0b-8edc-4182-996c-ab1dd114c1b5 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:12.156260 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 847/1691] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:01:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 11 12:01:12.172459 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f384c99a-46f9-4d47-8453-d79a3adb5902 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:12.173047 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f384c99a-46f9-4d47-8453-d79a3adb5902 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:12.174397 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f384c99a-46f9-4d47-8453-d79a3adb5902 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:12.211644 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-f384c99a-46f9-4d47-8453-d79a3adb5902 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:01:12.217403 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-f384c99a-46f9-4d47-8453-d79a3adb5902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:12.247071 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-1a031d0b-8edc-4182-996c-ab1dd114c1b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67c854dbc3024ce28885a7eb854c8de4', 'request_id': 'req-1a031d0b-8edc-4182-996c-ab1dd114c1b5'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:12.268093 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-1a031d0b-8edc-4182-996c-ab1dd114c1b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67c854dbc3024ce28885a7eb854c8de4', 'request_id': 'req-1a031d0b-8edc-4182-996c-ab1dd114c1b5'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:12.290134 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-1a031d0b-8edc-4182-996c-ab1dd114c1b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67c854dbc3024ce28885a7eb854c8de4', 'request_id': 'req-1a031d0b-8edc-4182-996c-ab1dd114c1b5'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:12.335645 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-1a031d0b-8edc-4182-996c-ab1dd114c1b5 admin admin] The token cache is being invalidate because project 67c854dbc3024ce28885a7eb854c8de4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 12:01:12.336054 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-1a031d0b-8edc-4182-996c-ab1dd114c1b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:12.338204 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-1a031d0b-8edc-4182-996c-ab1dd114c1b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:12.346642 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 845/1692] 10.222.0.59 () {58 vars in 1233 bytes} [Tue Oct 11 12:01:11 2022] DELETE /identity/v3/projects/67c854dbc3024ce28885a7eb854c8de4 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:01:12.355028 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a031d0b-8edc-4182-996c-ab1dd114c1b5 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:12.364572 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-1a031d0b-8edc-4182-996c-ab1dd114c1b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:12.483089 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:01:12.513266 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 848/1693] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:01:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 356 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:01:12.522956 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbed898-aa30-4447-b80d-78fb3c88e662 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:12.523536 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbed898-aa30-4447-b80d-78fb3c88e662 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:12.524042 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbed898-aa30-4447-b80d-78fb3c88e662 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:12.572099 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-4fbed898-aa30-4447-b80d-78fb3c88e662 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:01:12.575094 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-4fbed898-aa30-4447-b80d-78fb3c88e662 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:12.597362 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a031d0b-8edc-4182-996c-ab1dd114c1b5 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:12.605453 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2374f8-4b7f-4c45-9cf1-e40291771e0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:12.606185 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 849/1694] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:01:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:01:12.606897 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2374f8-4b7f-4c45-9cf1-e40291771e0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:12.607410 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2374f8-4b7f-4c45-9cf1-e40291771e0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:12.607904 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2374f8-4b7f-4c45-9cf1-e40291771e0c admin admin] PATH_INFO: `/v3/users/54a7fd945cdb4032bcb771c5d0f390ac` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:12.610186 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2374f8-4b7f-4c45-9cf1-e40291771e0c admin admin] RBAC: Authorizing `identity:delete_user(user_id=54a7fd945cdb4032bcb771c5d0f390ac)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:12.614191 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2374f8-4b7f-4c45-9cf1-e40291771e0c admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:12.617837 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbed898-aa30-4447-b80d-78fb3c88e662 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:12.642209 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbed898-aa30-4447-b80d-78fb3c88e662 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:12.652549 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff90b96-b5f3-4fe9-8fc6-6439a1188922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:12.654548 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff90b96-b5f3-4fe9-8fc6-6439a1188922 admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:12.655076 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff90b96-b5f3-4fe9-8fc6-6439a1188922 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:12.655554 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff90b96-b5f3-4fe9-8fc6-6439a1188922 admin admin] PATH_INFO: `/v3/domains` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:12.656815 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff90b96-b5f3-4fe9-8fc6-6439a1188922 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 12:01:12.657608 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff90b96-b5f3-4fe9-8fc6-6439a1188922 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:12.661023 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff90b96-b5f3-4fe9-8fc6-6439a1188922 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:12.714680 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 850/1695] 10.222.0.59 () {58 vars in 1153 bytes} [Tue Oct 11 12:01:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 12:01:12.715785 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-ef2374f8-4b7f-4c45-9cf1-e40291771e0c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54a7fd945cdb4032bcb771c5d0f390ac'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:12.715947 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff90b96-b5f3-4fe9-8fc6-6439a1188922 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:12.744154 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff90b96-b5f3-4fe9-8fc6-6439a1188922 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:12.759916 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a34d4e-e2f5-462d-aa1e-0b4bedaccc16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:12.761421 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a34d4e-e2f5-462d-aa1e-0b4bedaccc16 admin admin] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:12.761922 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a34d4e-e2f5-462d-aa1e-0b4bedaccc16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:12.762520 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a34d4e-e2f5-462d-aa1e-0b4bedaccc16 admin admin] PATH_INFO: `/v3/projects` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:12.763768 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a34d4e-e2f5-462d-aa1e-0b4bedaccc16 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:12.767938 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-ef2374f8-4b7f-4c45-9cf1-e40291771e0c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54a7fd945cdb4032bcb771c5d0f390ac'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:12.768696 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a34d4e-e2f5-462d-aa1e-0b4bedaccc16 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:12.786414 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-ef2374f8-4b7f-4c45-9cf1-e40291771e0c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54a7fd945cdb4032bcb771c5d0f390ac'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:12.788294 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-ef2374f8-4b7f-4c45-9cf1-e40291771e0c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54a7fd945cdb4032bcb771c5d0f390ac'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:12.807892 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-ef2374f8-4b7f-4c45-9cf1-e40291771e0c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54a7fd945cdb4032bcb771c5d0f390ac'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:12.823211 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 846/1696] 10.222.0.59 () {58 vars in 1224 bytes} [Tue Oct 11 12:01:12 2022] DELETE /identity/v3/users/54a7fd945cdb4032bcb771c5d0f390ac => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:01:12.828012 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2374f8-4b7f-4c45-9cf1-e40291771e0c admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:12.830589 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-ef2374f8-4b7f-4c45-9cf1-e40291771e0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:12.918676 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 851/1697] 10.222.0.59 () {60 vars in 1153 bytes} [Tue Oct 11 12:01:12 2022] POST /identity/v3/projects => generated 359 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:01:12.923924 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a34d4e-e2f5-462d-aa1e-0b4bedaccc16 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:12.940187 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-e7a34d4e-e2f5-462d-aa1e-0b4bedaccc16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:12.990451 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2374f8-4b7f-4c45-9cf1-e40291771e0c admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:13.043844 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:01:13.071917 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:01:13.074515 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a34d4e-e2f5-462d-aa1e-0b4bedaccc16 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:13.087265 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cc8fde-afcd-49e3-a13b-6dd596e5ef8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:13.087962 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cc8fde-afcd-49e3-a13b-6dd596e5ef8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:13.088436 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cc8fde-afcd-49e3-a13b-6dd596e5ef8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:13.088799 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cc8fde-afcd-49e3-a13b-6dd596e5ef8a admin admin] PATH_INFO: `/v3/projects/9fcb546ffbbe429fb588c7ae2bdddac8` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:13.089283 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c5060d-510e-4a1e-9462-7d34bee6a857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:13.090144 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cc8fde-afcd-49e3-a13b-6dd596e5ef8a admin admin] RBAC: Authorizing `identity:delete_project(project_id=9fcb546ffbbe429fb588c7ae2bdddac8)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:13.091401 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c5060d-510e-4a1e-9462-7d34bee6a857 admin admin] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:13.091781 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c5060d-510e-4a1e-9462-7d34bee6a857 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:13.092165 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c5060d-510e-4a1e-9462-7d34bee6a857 admin admin] PATH_INFO: `/v3/users` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:13.092817 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cc8fde-afcd-49e3-a13b-6dd596e5ef8a admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:13.093775 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c5060d-510e-4a1e-9462-7d34bee6a857 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:13.096817 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c5060d-510e-4a1e-9462-7d34bee6a857 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:13.132591 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 852/1698] 10.222.0.59 () {60 vars in 1144 bytes} [Tue Oct 11 12:01:12 2022] POST /identity/v3/users => generated 280 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:01:13.136467 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c5060d-510e-4a1e-9462-7d34bee6a857 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:13.150304 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c5060d-510e-4a1e-9462-7d34bee6a857 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:13.172414 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-63cc8fde-afcd-49e3-a13b-6dd596e5ef8a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9fcb546ffbbe429fb588c7ae2bdddac8', 'request_id': 'req-63cc8fde-afcd-49e3-a13b-6dd596e5ef8a'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:13.184816 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-63cc8fde-afcd-49e3-a13b-6dd596e5ef8a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9fcb546ffbbe429fb588c7ae2bdddac8', 'request_id': 'req-63cc8fde-afcd-49e3-a13b-6dd596e5ef8a'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:13.197352 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-63cc8fde-afcd-49e3-a13b-6dd596e5ef8a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9fcb546ffbbe429fb588c7ae2bdddac8', 'request_id': 'req-63cc8fde-afcd-49e3-a13b-6dd596e5ef8a'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:13.215137 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-63cc8fde-afcd-49e3-a13b-6dd596e5ef8a admin admin] The token cache is being invalidate because project 9fcb546ffbbe429fb588c7ae2bdddac8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 12:01:13.215446 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-63cc8fde-afcd-49e3-a13b-6dd596e5ef8a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:13.216406 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-63cc8fde-afcd-49e3-a13b-6dd596e5ef8a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:13.218196 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 847/1699] 10.222.0.59 () {58 vars in 1233 bytes} [Tue Oct 11 12:01:12 2022] DELETE /identity/v3/projects/9fcb546ffbbe429fb588c7ae2bdddac8 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:01:13.223058 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cc8fde-afcd-49e3-a13b-6dd596e5ef8a admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:13.224851 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-63cc8fde-afcd-49e3-a13b-6dd596e5ef8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:13.289626 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703be6b6-b1e0-4430-939a-864e8a58c50d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:13.291575 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703be6b6-b1e0-4430-939a-864e8a58c50d admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:13.292863 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703be6b6-b1e0-4430-939a-864e8a58c50d admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:13.294355 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703be6b6-b1e0-4430-939a-864e8a58c50d admin admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:13.295877 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703be6b6-b1e0-4430-939a-864e8a58c50d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:13.302371 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703be6b6-b1e0-4430-939a-864e8a58c50d admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:13.333159 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 853/1700] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:01:13 2022] GET /identity/v3/roles => generated 1420 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 11 12:01:13.339352 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703be6b6-b1e0-4430-939a-864e8a58c50d admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:13.346337 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-703be6b6-b1e0-4430-939a-864e8a58c50d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:13.362271 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cc8fde-afcd-49e3-a13b-6dd596e5ef8a admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:13.367648 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703be6b6-b1e0-4430-939a-864e8a58c50d admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:13.368038 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781757b4-0199-47a0-b14c-e572329ee1d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:13.368790 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781757b4-0199-47a0-b14c-e572329ee1d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:13.369080 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781757b4-0199-47a0-b14c-e572329ee1d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:13.369357 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781757b4-0199-47a0-b14c-e572329ee1d7 admin admin] PATH_INFO: `/v3/users/6cf3c328623c48e8bf1baa5384a4e465` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:13.370709 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781757b4-0199-47a0-b14c-e572329ee1d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6cf3c328623c48e8bf1baa5384a4e465)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:13.372954 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781757b4-0199-47a0-b14c-e572329ee1d7 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:13.376143 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a812334c-ed65-48cb-9dca-33889b3ce9b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:13.377756 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a812334c-ed65-48cb-9dca-33889b3ce9b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:13.379206 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a812334c-ed65-48cb-9dca-33889b3ce9b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:13.379526 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a812334c-ed65-48cb-9dca-33889b3ce9b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:13.380279 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a812334c-ed65-48cb-9dca-33889b3ce9b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:13.383015 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a812334c-ed65-48cb-9dca-33889b3ce9b3 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:13.410415 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 854/1701] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:01:13 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 11 12:01:13.413326 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a812334c-ed65-48cb-9dca-33889b3ce9b3 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:13.441463 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a812334c-ed65-48cb-9dca-33889b3ce9b3 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:13.480314 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc06542d-342a-437e-996c-56434c46532b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:13.481064 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc06542d-342a-437e-996c-56434c46532b admin admin] REQUEST_METHOD: `PUT` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:13.481378 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc06542d-342a-437e-996c-56434c46532b admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:13.481662 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc06542d-342a-437e-996c-56434c46532b admin admin] PATH_INFO: `/v3/projects/29ad40b0a95e4eb1a69c01de11261aa7/users/6f859fff66804fb4af5aa43f7745c664/roles/de2d2bd26fff4748a79990f97a107164` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:13.515739 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-781757b4-0199-47a0-b14c-e572329ee1d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6cf3c328623c48e8bf1baa5384a4e465'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:13.533350 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc06542d-342a-437e-996c-56434c46532b admin admin] RBAC: Authorizing `identity:create_grant(project_id=29ad40b0a95e4eb1a69c01de11261aa7, user_id=6f859fff66804fb4af5aa43f7745c664, role_id=de2d2bd26fff4748a79990f97a107164)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:13.540092 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc06542d-342a-437e-996c-56434c46532b admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:13.551082 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-781757b4-0199-47a0-b14c-e572329ee1d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6cf3c328623c48e8bf1baa5384a4e465'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:13.566895 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-781757b4-0199-47a0-b14c-e572329ee1d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6cf3c328623c48e8bf1baa5384a4e465'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:13.567319 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 855/1702] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 12:01:13 2022] PUT /identity/v3/projects/29ad40b0a95e4eb1a69c01de11261aa7/users/6f859fff66804fb4af5aa43f7745c664/roles/de2d2bd26fff4748a79990f97a107164 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:01:13.583206 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-781757b4-0199-47a0-b14c-e572329ee1d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6cf3c328623c48e8bf1baa5384a4e465'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:13.591744 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc06542d-342a-437e-996c-56434c46532b admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:13.597697 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-bc06542d-342a-437e-996c-56434c46532b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:13.606313 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-781757b4-0199-47a0-b14c-e572329ee1d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6cf3c328623c48e8bf1baa5384a4e465'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:13.614455 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:01:13.643202 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 848/1703] 10.222.0.59 () {58 vars in 1224 bytes} [Tue Oct 11 12:01:13 2022] DELETE /identity/v3/users/6cf3c328623c48e8bf1baa5384a4e465 => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:01:13.650101 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c316c5-dd2c-44e6-9e2b-a457c740705a None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:13.651774 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c316c5-dd2c-44e6-9e2b-a457c740705a None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:13.654226 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c316c5-dd2c-44e6-9e2b-a457c740705a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:13.717626 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc06542d-342a-437e-996c-56434c46532b admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:13.755059 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-54c316c5-dd2c-44e6-9e2b-a457c740705a None None] MFA Rules not processed for user `6f859fff66804fb4af5aa43f7745c664`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:01:13.756471 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-54c316c5-dd2c-44e6-9e2b-a457c740705a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:13.817499 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:01:13.858506 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568d1bd5-cd29-4b9e-ba89-3c699e097327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:13.861563 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568d1bd5-cd29-4b9e-ba89-3c699e097327 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:13.863034 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568d1bd5-cd29-4b9e-ba89-3c699e097327 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:13.866098 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568d1bd5-cd29-4b9e-ba89-3c699e097327 admin admin] PATH_INFO: `/v3/projects/6a57fc9be1b74695a3d87e8d2f999296` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:13.868888 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568d1bd5-cd29-4b9e-ba89-3c699e097327 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a57fc9be1b74695a3d87e8d2f999296)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:13.878003 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568d1bd5-cd29-4b9e-ba89-3c699e097327 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:13.879670 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 849/1704] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:01:13 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:01:13.880760 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c316c5-dd2c-44e6-9e2b-a457c740705a None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:13.882804 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-54c316c5-dd2c-44e6-9e2b-a457c740705a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:13.901872 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c316c5-dd2c-44e6-9e2b-a457c740705a None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:13.909686 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18021cec-e57a-4841-8565-440ba8eb9434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:13.910359 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18021cec-e57a-4841-8565-440ba8eb9434 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:13.911112 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18021cec-e57a-4841-8565-440ba8eb9434 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:13.911403 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18021cec-e57a-4841-8565-440ba8eb9434 admin admin] PATH_INFO: `/v3/projects/b0a8be5046d3472e9d5efdee0931b887` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:13.912710 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18021cec-e57a-4841-8565-440ba8eb9434 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0a8be5046d3472e9d5efdee0931b887)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:13.914890 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18021cec-e57a-4841-8565-440ba8eb9434 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:14.003234 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-568d1bd5-cd29-4b9e-ba89-3c699e097327 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a57fc9be1b74695a3d87e8d2f999296', 'request_id': 'req-568d1bd5-cd29-4b9e-ba89-3c699e097327'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:14.007843 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-18021cec-e57a-4841-8565-440ba8eb9434 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0a8be5046d3472e9d5efdee0931b887', 'request_id': 'req-18021cec-e57a-4841-8565-440ba8eb9434'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:14.015974 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-18021cec-e57a-4841-8565-440ba8eb9434 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0a8be5046d3472e9d5efdee0931b887', 'request_id': 'req-18021cec-e57a-4841-8565-440ba8eb9434'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:14.018937 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-568d1bd5-cd29-4b9e-ba89-3c699e097327 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a57fc9be1b74695a3d87e8d2f999296', 'request_id': 'req-568d1bd5-cd29-4b9e-ba89-3c699e097327'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:14.029366 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-18021cec-e57a-4841-8565-440ba8eb9434 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0a8be5046d3472e9d5efdee0931b887', 'request_id': 'req-18021cec-e57a-4841-8565-440ba8eb9434'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:14.034684 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-568d1bd5-cd29-4b9e-ba89-3c699e097327 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a57fc9be1b74695a3d87e8d2f999296', 'request_id': 'req-568d1bd5-cd29-4b9e-ba89-3c699e097327'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:14.047424 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-18021cec-e57a-4841-8565-440ba8eb9434 admin admin] The token cache is being invalidate because project b0a8be5046d3472e9d5efdee0931b887 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 12:01:14.048027 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-18021cec-e57a-4841-8565-440ba8eb9434 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:14.049329 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-18021cec-e57a-4841-8565-440ba8eb9434 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:14.051233 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 850/1705] 10.222.0.59 () {58 vars in 1233 bytes} [Tue Oct 11 12:01:13 2022] DELETE /identity/v3/projects/b0a8be5046d3472e9d5efdee0931b887 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:01:14.052279 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18021cec-e57a-4841-8565-440ba8eb9434 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:14.053848 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-18021cec-e57a-4841-8565-440ba8eb9434 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:14.068418 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-568d1bd5-cd29-4b9e-ba89-3c699e097327 admin admin] The token cache is being invalidate because project 6a57fc9be1b74695a3d87e8d2f999296 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81432) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 12:01:14.069440 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-568d1bd5-cd29-4b9e-ba89-3c699e097327 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:14.071179 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-568d1bd5-cd29-4b9e-ba89-3c699e097327 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:14.074019 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 856/1706] 10.222.0.59 () {58 vars in 1233 bytes} [Tue Oct 11 12:01:13 2022] DELETE /identity/v3/projects/6a57fc9be1b74695a3d87e8d2f999296 => generated 0 bytes in 506 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:01:14.076591 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978b0487-8929-44f5-8604-44b2af272307 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:14.077440 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978b0487-8929-44f5-8604-44b2af272307 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:14.078007 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978b0487-8929-44f5-8604-44b2af272307 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:14.126077 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-978b0487-8929-44f5-8604-44b2af272307 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:01:14.128536 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-978b0487-8929-44f5-8604-44b2af272307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:14.168589 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18021cec-e57a-4841-8565-440ba8eb9434 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:14.191749 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02a6d35-c5fa-43e2-a847-823ec12bc7a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b1b651c73664d08accc6edf2f79bace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:14.192483 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02a6d35-c5fa-43e2-a847-823ec12bc7a4 service cinder] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:14.192770 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02a6d35-c5fa-43e2-a847-823ec12bc7a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:14.193150 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02a6d35-c5fa-43e2-a847-823ec12bc7a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:14.195358 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-c02a6d35-c5fa-43e2-a847-823ec12bc7a4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:14.234878 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:01:14.316217 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02a6d35-c5fa-43e2-a847-823ec12bc7a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:14.317928 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02a6d35-c5fa-43e2-a847-823ec12bc7a4 service cinder] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:14.325646 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 857/1707] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:01:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:01:14.327695 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978b0487-8929-44f5-8604-44b2af272307 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:14.336540 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-978b0487-8929-44f5-8604-44b2af272307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:14.338483 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 851/1708] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:01:14 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:01:14.340126 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76576127-8955-4d1b-8d1e-b57fd97c6567 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:14.340530 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76576127-8955-4d1b-8d1e-b57fd97c6567 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:14.340982 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76576127-8955-4d1b-8d1e-b57fd97c6567 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:14.374863 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-76576127-8955-4d1b-8d1e-b57fd97c6567 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:01:14.376168 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978b0487-8929-44f5-8604-44b2af272307 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:14.376411 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-76576127-8955-4d1b-8d1e-b57fd97c6567 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:14.396559 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b72879-5bad-4385-b33d-0dd9bd4ef53e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:14.398320 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b72879-5bad-4385-b33d-0dd9bd4ef53e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:14.398320 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b72879-5bad-4385-b33d-0dd9bd4ef53e admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:14.398661 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b72879-5bad-4385-b33d-0dd9bd4ef53e admin admin] PATH_INFO: `/v3/users/bafe8313685e462b94055bcd6bbd5259` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:14.402514 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 852/1709] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:01:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:01:14.404497 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b72879-5bad-4385-b33d-0dd9bd4ef53e admin admin] RBAC: Authorizing `identity:delete_user(user_id=bafe8313685e462b94055bcd6bbd5259)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:14.408228 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76576127-8955-4d1b-8d1e-b57fd97c6567 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:14.411138 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-76576127-8955-4d1b-8d1e-b57fd97c6567 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:14.411636 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b72879-5bad-4385-b33d-0dd9bd4ef53e admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:14.434864 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76576127-8955-4d1b-8d1e-b57fd97c6567 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:14.456176 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7b2002-641f-4b70-b01c-302394b22374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:14.456924 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7b2002-641f-4b70-b01c-302394b22374 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:14.457207 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7b2002-641f-4b70-b01c-302394b22374 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:14.457483 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7b2002-641f-4b70-b01c-302394b22374 admin admin] PATH_INFO: `/v3/users/6f859fff66804fb4af5aa43f7745c664` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:14.459219 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7b2002-641f-4b70-b01c-302394b22374 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f859fff66804fb4af5aa43f7745c664)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:14.461297 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7b2002-641f-4b70-b01c-302394b22374 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:14.549091 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-76b72879-5bad-4385-b33d-0dd9bd4ef53e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bafe8313685e462b94055bcd6bbd5259'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:14.550179 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-ae7b2002-641f-4b70-b01c-302394b22374 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f859fff66804fb4af5aa43f7745c664'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:14.569550 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-ae7b2002-641f-4b70-b01c-302394b22374 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f859fff66804fb4af5aa43f7745c664'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:14.583444 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-ae7b2002-641f-4b70-b01c-302394b22374 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f859fff66804fb4af5aa43f7745c664'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:14.584396 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-ae7b2002-641f-4b70-b01c-302394b22374 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f859fff66804fb4af5aa43f7745c664'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:14.596866 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-ae7b2002-641f-4b70-b01c-302394b22374 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f859fff66804fb4af5aa43f7745c664'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:14.597077 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-76b72879-5bad-4385-b33d-0dd9bd4ef53e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bafe8313685e462b94055bcd6bbd5259'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:14.598260 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-76b72879-5bad-4385-b33d-0dd9bd4ef53e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bafe8313685e462b94055bcd6bbd5259'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:14.601456 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:01:14.607224 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 853/1710] 10.222.0.59 () {58 vars in 1224 bytes} [Tue Oct 11 12:01:14 2022] DELETE /identity/v3/users/6f859fff66804fb4af5aa43f7745c664 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:01:14.609091 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7b2002-641f-4b70-b01c-302394b22374 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:14.611087 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-ae7b2002-641f-4b70-b01c-302394b22374 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:14.616887 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-76b72879-5bad-4385-b33d-0dd9bd4ef53e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bafe8313685e462b94055bcd6bbd5259'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:14.619108 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-76b72879-5bad-4385-b33d-0dd9bd4ef53e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bafe8313685e462b94055bcd6bbd5259'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:14.650522 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 858/1711] 10.222.0.59 () {58 vars in 1224 bytes} [Tue Oct 11 12:01:14 2022] DELETE /identity/v3/users/bafe8313685e462b94055bcd6bbd5259 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:01:14.652818 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b72879-5bad-4385-b33d-0dd9bd4ef53e admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:14.656536 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-76b72879-5bad-4385-b33d-0dd9bd4ef53e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:14.713967 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7b2002-641f-4b70-b01c-302394b22374 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:14.779140 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:01:14.810450 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b72879-5bad-4385-b33d-0dd9bd4ef53e admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:14.822809 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcfba93-efda-4d10-b341-2c6261f2ea4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:14.824311 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcfba93-efda-4d10-b341-2c6261f2ea4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:14.824955 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcfba93-efda-4d10-b341-2c6261f2ea4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:14.825723 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcfba93-efda-4d10-b341-2c6261f2ea4b admin admin] PATH_INFO: `/v3/projects/96244b408c054f64bdfd02a5c994ace3` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:14.828517 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcfba93-efda-4d10-b341-2c6261f2ea4b admin admin] RBAC: Authorizing `identity:delete_project(project_id=96244b408c054f64bdfd02a5c994ace3)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:14.833175 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcfba93-efda-4d10-b341-2c6261f2ea4b admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:14.835450 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bde8579-b8b8-4ea0-b272-9a7af3070e74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:14.837603 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bde8579-b8b8-4ea0-b272-9a7af3070e74 admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:14.838087 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bde8579-b8b8-4ea0-b272-9a7af3070e74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:14.841190 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bde8579-b8b8-4ea0-b272-9a7af3070e74 admin admin] PATH_INFO: `/v3/domains` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:14.842249 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bde8579-b8b8-4ea0-b272-9a7af3070e74 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 12:01:14.842874 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bde8579-b8b8-4ea0-b272-9a7af3070e74 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:14.849096 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bde8579-b8b8-4ea0-b272-9a7af3070e74 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:14.884649 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 854/1712] 10.222.0.59 () {58 vars in 1153 bytes} [Tue Oct 11 12:01:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 12:01:14.884649 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b614c93d-1640-4dee-afbd-e190fc4b50b8 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:14.884649 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b614c93d-1640-4dee-afbd-e190fc4b50b8 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:14.892546 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b614c93d-1640-4dee-afbd-e190fc4b50b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:14.942853 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-b614c93d-1640-4dee-afbd-e190fc4b50b8 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:01:14.945505 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-b614c93d-1640-4dee-afbd-e190fc4b50b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:14.968348 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-bbcfba93-efda-4d10-b341-2c6261f2ea4b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96244b408c054f64bdfd02a5c994ace3', 'request_id': 'req-bbcfba93-efda-4d10-b341-2c6261f2ea4b'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:14.988860 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-bbcfba93-efda-4d10-b341-2c6261f2ea4b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96244b408c054f64bdfd02a5c994ace3', 'request_id': 'req-bbcfba93-efda-4d10-b341-2c6261f2ea4b'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:15.019778 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-bbcfba93-efda-4d10-b341-2c6261f2ea4b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96244b408c054f64bdfd02a5c994ace3', 'request_id': 'req-bbcfba93-efda-4d10-b341-2c6261f2ea4b'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:15.041907 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-bbcfba93-efda-4d10-b341-2c6261f2ea4b admin admin] The token cache is being invalidate because project 96244b408c054f64bdfd02a5c994ace3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81432) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 12:01:15.042162 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-bbcfba93-efda-4d10-b341-2c6261f2ea4b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:15.043438 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-bbcfba93-efda-4d10-b341-2c6261f2ea4b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:15.045500 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 859/1713] 10.222.0.59 () {58 vars in 1233 bytes} [Tue Oct 11 12:01:14 2022] DELETE /identity/v3/projects/96244b408c054f64bdfd02a5c994ace3 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:01:15.046411 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcfba93-efda-4d10-b341-2c6261f2ea4b admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:15.048026 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-bbcfba93-efda-4d10-b341-2c6261f2ea4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:15.123720 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 855/1714] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:01:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:01:15.125143 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5330db0-fe94-4124-9f81-767f7a73f265 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:15.125440 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5330db0-fe94-4124-9f81-767f7a73f265 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:15.125717 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5330db0-fe94-4124-9f81-767f7a73f265 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:15.160742 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-b5330db0-fe94-4124-9f81-767f7a73f265 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:01:15.162190 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-b5330db0-fe94-4124-9f81-767f7a73f265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:15.174190 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 856/1715] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:01:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:01:15.175237 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5330db0-fe94-4124-9f81-767f7a73f265 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:15.175612 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcfba93-efda-4d10-b341-2c6261f2ea4b admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:15.177559 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-b5330db0-fe94-4124-9f81-767f7a73f265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:15.191333 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642552e3-b47d-4042-bb91-6f77c5105670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:15.192336 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642552e3-b47d-4042-bb91-6f77c5105670 admin admin] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:15.192682 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642552e3-b47d-4042-bb91-6f77c5105670 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:15.193021 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642552e3-b47d-4042-bb91-6f77c5105670 admin admin] PATH_INFO: `/v3/projects` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:15.193879 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642552e3-b47d-4042-bb91-6f77c5105670 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:15.196993 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642552e3-b47d-4042-bb91-6f77c5105670 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:15.229710 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:INSERT=3,keystone:DELETE=8,keystone:SELECT=89 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:01:15.248970 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 860/1716] 10.222.0.59 () {60 vars in 1153 bytes} [Tue Oct 11 12:01:15 2022] POST /identity/v3/projects => generated 383 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:01:15.251526 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d9c15c-109b-43c9-9cfc-f95b6bf5126d None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:15.252674 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d9c15c-109b-43c9-9cfc-f95b6bf5126d None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:15.253039 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d9c15c-109b-43c9-9cfc-f95b6bf5126d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:15.278530 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5330db0-fe94-4124-9f81-767f7a73f265 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:15.292732 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-f4d9c15c-109b-43c9-9cfc-f95b6bf5126d None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:01:15.294987 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-f4d9c15c-109b-43c9-9cfc-f95b6bf5126d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:15.385130 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1399b150-0ccf-418f-b2f3-6992950f2756 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e563fe174db4cc19ae6bf8162cc3d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:15.385788 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1399b150-0ccf-418f-b2f3-6992950f2756 service neutron] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:15.386070 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1399b150-0ccf-418f-b2f3-6992950f2756 service neutron] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:15.386346 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1399b150-0ccf-418f-b2f3-6992950f2756 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:15.460563 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 861/1717] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:01:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:01:15.464447 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:01:15.467061 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d9c15c-109b-43c9-9cfc-f95b6bf5126d None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:15.488867 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d9c15c-109b-43c9-9cfc-f95b6bf5126d None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:15.497212 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542859d3-2488-4a76-bc7b-eed026770640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:15.498132 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542859d3-2488-4a76-bc7b-eed026770640 admin admin] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:15.498454 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542859d3-2488-4a76-bc7b-eed026770640 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:15.498834 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542859d3-2488-4a76-bc7b-eed026770640 admin admin] PATH_INFO: `/v3/users` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:15.500479 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542859d3-2488-4a76-bc7b-eed026770640 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:15.503514 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542859d3-2488-4a76-bc7b-eed026770640 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:15.505727 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1399b150-0ccf-418f-b2f3-6992950f2756 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:15.507518 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1399b150-0ccf-418f-b2f3-6992950f2756 service neutron] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:15.520307 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 857/1718] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:01:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:01:15.521590 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1399b150-0ccf-418f-b2f3-6992950f2756 service neutron] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:15.524480 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-1399b150-0ccf-418f-b2f3-6992950f2756 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:15.540259 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1399b150-0ccf-418f-b2f3-6992950f2756 service neutron] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:15.546957 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 862/1719] 10.222.0.59 () {60 vars in 1144 bytes} [Tue Oct 11 12:01:15 2022] POST /identity/v3/users => generated 292 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:01:15.546957 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542859d3-2488-4a76-bc7b-eed026770640 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:15.551102 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1166dda0-76fd-40b3-b4f7-e0a26a2e06c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:15.552215 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1166dda0-76fd-40b3-b4f7-e0a26a2e06c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:15.552689 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1166dda0-76fd-40b3-b4f7-e0a26a2e06c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:15.553596 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1166dda0-76fd-40b3-b4f7-e0a26a2e06c2 admin admin] PATH_INFO: `/v3/projects/440847454347404aa5604cc949ebabe5` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:15.556600 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1166dda0-76fd-40b3-b4f7-e0a26a2e06c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=440847454347404aa5604cc949ebabe5)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:15.560605 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1166dda0-76fd-40b3-b4f7-e0a26a2e06c2 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:15.562215 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542859d3-2488-4a76-bc7b-eed026770640 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:15.569422 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720c07aa-ef74-4f59-bb74-9a57161051d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:15.570617 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720c07aa-ef74-4f59-bb74-9a57161051d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:15.571131 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720c07aa-ef74-4f59-bb74-9a57161051d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:15.571638 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720c07aa-ef74-4f59-bb74-9a57161051d1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:15.572913 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720c07aa-ef74-4f59-bb74-9a57161051d1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 12:01:15.573752 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720c07aa-ef74-4f59-bb74-9a57161051d1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:15.577580 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720c07aa-ef74-4f59-bb74-9a57161051d1 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:15.609561 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 863/1720] 10.222.0.59 () {58 vars in 1153 bytes} [Tue Oct 11 12:01:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 12:01:15.612074 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720c07aa-ef74-4f59-bb74-9a57161051d1 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:15.630765 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720c07aa-ef74-4f59-bb74-9a57161051d1 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:15.637277 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fff55d-875b-4c74-aa4f-135e23cf31d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:15.638309 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fff55d-875b-4c74-aa4f-135e23cf31d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:15.638822 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fff55d-875b-4c74-aa4f-135e23cf31d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:15.639432 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fff55d-875b-4c74-aa4f-135e23cf31d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:15.640316 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fff55d-875b-4c74-aa4f-135e23cf31d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:15.642666 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fff55d-875b-4c74-aa4f-135e23cf31d9 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:15.659959 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 864/1721] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:01:15 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:01:15.662213 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fff55d-875b-4c74-aa4f-135e23cf31d9 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:15.676812 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fff55d-875b-4c74-aa4f-135e23cf31d9 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:15.678886 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-1166dda0-76fd-40b3-b4f7-e0a26a2e06c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '440847454347404aa5604cc949ebabe5', 'request_id': 'req-1166dda0-76fd-40b3-b4f7-e0a26a2e06c2'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:15.694841 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-1166dda0-76fd-40b3-b4f7-e0a26a2e06c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '440847454347404aa5604cc949ebabe5', 'request_id': 'req-1166dda0-76fd-40b3-b4f7-e0a26a2e06c2'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:15.712455 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-1166dda0-76fd-40b3-b4f7-e0a26a2e06c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '440847454347404aa5604cc949ebabe5', 'request_id': 'req-1166dda0-76fd-40b3-b4f7-e0a26a2e06c2'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:15.742375 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-1166dda0-76fd-40b3-b4f7-e0a26a2e06c2 admin admin] The token cache is being invalidate because project 440847454347404aa5604cc949ebabe5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 12:01:15.742961 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-1166dda0-76fd-40b3-b4f7-e0a26a2e06c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:15.743998 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-1166dda0-76fd-40b3-b4f7-e0a26a2e06c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:15.746038 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 858/1722] 10.222.0.59 () {58 vars in 1233 bytes} [Tue Oct 11 12:01:15 2022] DELETE /identity/v3/projects/440847454347404aa5604cc949ebabe5 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:01:15.752033 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1166dda0-76fd-40b3-b4f7-e0a26a2e06c2 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:15.753758 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-1166dda0-76fd-40b3-b4f7-e0a26a2e06c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:15.829205 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c58602-88a0-4a4f-845b-23cce2c38526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:15.830821 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c58602-88a0-4a4f-845b-23cce2c38526 admin admin] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:15.831863 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c58602-88a0-4a4f-845b-23cce2c38526 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:15.832589 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c58602-88a0-4a4f-845b-23cce2c38526 admin admin] PATH_INFO: `/v3/projects` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:15.835334 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c58602-88a0-4a4f-845b-23cce2c38526 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:15.840232 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c58602-88a0-4a4f-845b-23cce2c38526 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:15.869573 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1166dda0-76fd-40b3-b4f7-e0a26a2e06c2 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:15.878245 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc65f178-dd32-4637-b10b-f934ec1ac5ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:15.879325 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc65f178-dd32-4637-b10b-f934ec1ac5ed admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:15.880009 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc65f178-dd32-4637-b10b-f934ec1ac5ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:15.880836 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc65f178-dd32-4637-b10b-f934ec1ac5ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:15.881604 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc65f178-dd32-4637-b10b-f934ec1ac5ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:15.884711 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc65f178-dd32-4637-b10b-f934ec1ac5ed admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:15.900346 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 865/1723] 10.222.0.59 () {60 vars in 1153 bytes} [Tue Oct 11 12:01:15 2022] POST /identity/v3/projects => generated 393 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:01:15.903357 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c7c2af-ad91-4ce2-8160-582cbd438a2a None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:15.903952 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c7c2af-ad91-4ce2-8160-582cbd438a2a None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:15.904361 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c7c2af-ad91-4ce2-8160-582cbd438a2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:15.917874 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 859/1724] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:01:15 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:01:15.919628 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc65f178-dd32-4637-b10b-f934ec1ac5ed admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:15.922418 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-bc65f178-dd32-4637-b10b-f934ec1ac5ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:15.949816 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc65f178-dd32-4637-b10b-f934ec1ac5ed admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:15.974769 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-09c7c2af-ad91-4ce2-8160-582cbd438a2a None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:01:15.979746 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-09c7c2af-ad91-4ce2-8160-582cbd438a2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:16.101710 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:01:16.105073 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:01:16.126888 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c044e0-0762-4436-bf08-617e41204927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:16.129970 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c044e0-0762-4436-bf08-617e41204927 admin admin] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:16.130576 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c044e0-0762-4436-bf08-617e41204927 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:16.131311 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c044e0-0762-4436-bf08-617e41204927 admin admin] PATH_INFO: `/v3/users` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:16.134258 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c044e0-0762-4436-bf08-617e41204927 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:16.140093 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c044e0-0762-4436-bf08-617e41204927 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:16.163709 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 866/1725] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:01:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:01:16.166182 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c7c2af-ad91-4ce2-8160-582cbd438a2a None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:16.186643 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c7c2af-ad91-4ce2-8160-582cbd438a2a None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:16.195713 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0049bdf3-e5b1-42ff-9158-c21fc71af866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:16.196688 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0049bdf3-e5b1-42ff-9158-c21fc71af866 admin admin] REQUEST_METHOD: `PUT` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:16.197227 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0049bdf3-e5b1-42ff-9158-c21fc71af866 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:16.198655 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0049bdf3-e5b1-42ff-9158-c21fc71af866 admin admin] PATH_INFO: `/v3/projects/4fef4ffc374a4e72bfffbb9e1d6411fa/users/05b7d4e8b9e0435391d9db79e5d68d5a/roles/de2d2bd26fff4748a79990f97a107164` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:16.203552 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 860/1726] 10.222.0.59 () {60 vars in 1144 bytes} [Tue Oct 11 12:01:15 2022] POST /identity/v3/users => generated 297 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:01:16.208436 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c044e0-0762-4436-bf08-617e41204927 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:16.212982 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-35c044e0-0762-4436-bf08-617e41204927 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:16.238642 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c044e0-0762-4436-bf08-617e41204927 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:16.240569 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0049bdf3-e5b1-42ff-9158-c21fc71af866 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4fef4ffc374a4e72bfffbb9e1d6411fa, user_id=05b7d4e8b9e0435391d9db79e5d68d5a, role_id=de2d2bd26fff4748a79990f97a107164)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:16.246986 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0049bdf3-e5b1-42ff-9158-c21fc71af866 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:16.249214 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ddfc59-af81-44ce-b3ea-ffdb5403fa4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:16.250145 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ddfc59-af81-44ce-b3ea-ffdb5403fa4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:16.250768 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ddfc59-af81-44ce-b3ea-ffdb5403fa4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:16.251490 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ddfc59-af81-44ce-b3ea-ffdb5403fa4e admin admin] PATH_INFO: `/v3/projects/29ad40b0a95e4eb1a69c01de11261aa7` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:16.253776 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ddfc59-af81-44ce-b3ea-ffdb5403fa4e admin admin] RBAC: Authorizing `identity:delete_project(project_id=29ad40b0a95e4eb1a69c01de11261aa7)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:16.256539 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ddfc59-af81-44ce-b3ea-ffdb5403fa4e admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:16.277460 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 867/1727] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 12:01:16 2022] PUT /identity/v3/projects/4fef4ffc374a4e72bfffbb9e1d6411fa/users/05b7d4e8b9e0435391d9db79e5d68d5a/roles/de2d2bd26fff4748a79990f97a107164 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:01:16.280207 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998a5a4e-9fbe-4b68-b241-cd08a256b948 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:16.280657 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998a5a4e-9fbe-4b68-b241-cd08a256b948 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:16.281312 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998a5a4e-9fbe-4b68-b241-cd08a256b948 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:16.321634 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-998a5a4e-9fbe-4b68-b241-cd08a256b948 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:01:16.324283 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-998a5a4e-9fbe-4b68-b241-cd08a256b948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:16.344076 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-44ddfc59-af81-44ce-b3ea-ffdb5403fa4e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '29ad40b0a95e4eb1a69c01de11261aa7', 'request_id': 'req-44ddfc59-af81-44ce-b3ea-ffdb5403fa4e'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:16.353953 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-44ddfc59-af81-44ce-b3ea-ffdb5403fa4e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '29ad40b0a95e4eb1a69c01de11261aa7', 'request_id': 'req-44ddfc59-af81-44ce-b3ea-ffdb5403fa4e'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:16.367377 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-44ddfc59-af81-44ce-b3ea-ffdb5403fa4e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '29ad40b0a95e4eb1a69c01de11261aa7', 'request_id': 'req-44ddfc59-af81-44ce-b3ea-ffdb5403fa4e'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:16.396420 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-44ddfc59-af81-44ce-b3ea-ffdb5403fa4e admin admin] The token cache is being invalidate because project 29ad40b0a95e4eb1a69c01de11261aa7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 12:01:16.396841 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-44ddfc59-af81-44ce-b3ea-ffdb5403fa4e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:16.398002 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-44ddfc59-af81-44ce-b3ea-ffdb5403fa4e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:16.400793 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 861/1728] 10.222.0.59 () {58 vars in 1233 bytes} [Tue Oct 11 12:01:16 2022] DELETE /identity/v3/projects/29ad40b0a95e4eb1a69c01de11261aa7 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:01:16.402491 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ddfc59-af81-44ce-b3ea-ffdb5403fa4e admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:16.409506 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-44ddfc59-af81-44ce-b3ea-ffdb5403fa4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:16.450925 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 868/1729] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:01:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:01:16.453568 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf72bad-d09d-4f9a-9ccd-6cefc4ad52a0 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:16.454034 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf72bad-d09d-4f9a-9ccd-6cefc4ad52a0 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:16.454515 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf72bad-d09d-4f9a-9ccd-6cefc4ad52a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:16.493560 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-1cf72bad-d09d-4f9a-9ccd-6cefc4ad52a0 None None] MFA Rules not processed for user `05b7d4e8b9e0435391d9db79e5d68d5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:01:16.494858 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-1cf72bad-d09d-4f9a-9ccd-6cefc4ad52a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:16.513554 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ddfc59-af81-44ce-b3ea-ffdb5403fa4e admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:16.519693 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c867edf2-668a-4540-9054-5cb5645ed736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:16.520481 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c867edf2-668a-4540-9054-5cb5645ed736 admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:16.520768 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c867edf2-668a-4540-9054-5cb5645ed736 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:16.521038 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c867edf2-668a-4540-9054-5cb5645ed736 admin admin] PATH_INFO: `/v3/roles` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:16.521712 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c867edf2-668a-4540-9054-5cb5645ed736 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:16.523428 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c867edf2-668a-4540-9054-5cb5645ed736 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:16.535609 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 862/1730] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:01:16 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:01:16.537271 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483d9a0e-703a-4dff-8794-9c0ead2eb0b0 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:16.537271 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483d9a0e-703a-4dff-8794-9c0ead2eb0b0 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:16.537516 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483d9a0e-703a-4dff-8794-9c0ead2eb0b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:16.559619 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-483d9a0e-703a-4dff-8794-9c0ead2eb0b0 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:01:16.561030 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-483d9a0e-703a-4dff-8794-9c0ead2eb0b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:16.569578 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 863/1731] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:01:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:01:16.577136 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483d9a0e-703a-4dff-8794-9c0ead2eb0b0 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:16.595997 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483d9a0e-703a-4dff-8794-9c0ead2eb0b0 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:16.607641 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8216d96c-c990-4f39-af95-40ac602e1751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:16.608662 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8216d96c-c990-4f39-af95-40ac602e1751 admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:16.609181 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8216d96c-c990-4f39-af95-40ac602e1751 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:16.610542 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8216d96c-c990-4f39-af95-40ac602e1751 admin admin] PATH_INFO: `/v3/domains` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:16.611934 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8216d96c-c990-4f39-af95-40ac602e1751 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 12:01:16.612434 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8216d96c-c990-4f39-af95-40ac602e1751 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:16.616482 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8216d96c-c990-4f39-af95-40ac602e1751 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:16.626324 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 869/1732] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:01:16 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:01:16.626324 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf72bad-d09d-4f9a-9ccd-6cefc4ad52a0 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:16.644389 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 864/1733] 10.222.0.59 () {58 vars in 1153 bytes} [Tue Oct 11 12:01:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 12:01:16.644389 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630504fd-1d0c-4585-bcdd-3da40b4f8ddd None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:16.644389 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630504fd-1d0c-4585-bcdd-3da40b4f8ddd None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:16.644389 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630504fd-1d0c-4585-bcdd-3da40b4f8ddd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:16.649597 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf72bad-d09d-4f9a-9ccd-6cefc4ad52a0 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:16.659024 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd178ef8-8596-4b11-8657-f8c821ed9037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:16.660219 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd178ef8-8596-4b11-8657-f8c821ed9037 admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:16.661027 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd178ef8-8596-4b11-8657-f8c821ed9037 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:16.661807 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd178ef8-8596-4b11-8657-f8c821ed9037 admin admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:16.663109 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd178ef8-8596-4b11-8657-f8c821ed9037 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:16.666348 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd178ef8-8596-4b11-8657-f8c821ed9037 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:16.669807 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-630504fd-1d0c-4585-bcdd-3da40b4f8ddd None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:01:16.671726 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-630504fd-1d0c-4585-bcdd-3da40b4f8ddd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:16.685577 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 865/1734] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:01:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:01:16.685577 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630504fd-1d0c-4585-bcdd-3da40b4f8ddd None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:16.687592 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-630504fd-1d0c-4585-bcdd-3da40b4f8ddd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:16.692230 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 870/1735] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:01:16 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:01:16.692230 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd178ef8-8596-4b11-8657-f8c821ed9037 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:16.701959 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630504fd-1d0c-4585-bcdd-3da40b4f8ddd None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:16.707710 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67b4bf7-9cb8-4557-8752-6ffd30c785c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:16.708374 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67b4bf7-9cb8-4557-8752-6ffd30c785c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:16.708717 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67b4bf7-9cb8-4557-8752-6ffd30c785c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:16.709056 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67b4bf7-9cb8-4557-8752-6ffd30c785c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:16.709789 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67b4bf7-9cb8-4557-8752-6ffd30c785c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:16.711383 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd178ef8-8596-4b11-8657-f8c821ed9037 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:16.711948 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67b4bf7-9cb8-4557-8752-6ffd30c785c5 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:16.719585 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b8660c-2f85-45ec-a99e-2715dc73d1c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:16.720674 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b8660c-2f85-45ec-a99e-2715dc73d1c4 admin admin] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:16.721156 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b8660c-2f85-45ec-a99e-2715dc73d1c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:16.721636 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b8660c-2f85-45ec-a99e-2715dc73d1c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:16.722843 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b8660c-2f85-45ec-a99e-2715dc73d1c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:16.726665 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b8660c-2f85-45ec-a99e-2715dc73d1c4 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:16.753098 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 866/1736] 10.222.0.59 () {60 vars in 1153 bytes} [Tue Oct 11 12:01:16 2022] POST /identity/v3/projects => generated 383 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:01:16.753098 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67b4bf7-9cb8-4557-8752-6ffd30c785c5 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:16.772478 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 871/1737] 10.222.0.59 () {60 vars in 1153 bytes} [Tue Oct 11 12:01:16 2022] POST /identity/v3/projects => generated 365 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:01:16.772478 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b8660c-2f85-45ec-a99e-2715dc73d1c4 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:16.782052 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:01:16.807751 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:01:16.860064 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67b4bf7-9cb8-4557-8752-6ffd30c785c5 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:16.904569 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b8660c-2f85-45ec-a99e-2715dc73d1c4 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:16.912216 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2110333b-9d10-4045-9f70-0257c18dc133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:16.912922 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2110333b-9d10-4045-9f70-0257c18dc133 admin admin] REQUEST_METHOD: `PUT` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:16.913196 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2110333b-9d10-4045-9f70-0257c18dc133 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:16.913469 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2110333b-9d10-4045-9f70-0257c18dc133 admin admin] PATH_INFO: `/v3/projects/21b141f7b6394853b423e3366c8e8692/users/4905b1221a0d44d99c8f0e739e4a8c94/roles/de2d2bd26fff4748a79990f97a107164` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:16.935002 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2110333b-9d10-4045-9f70-0257c18dc133 admin admin] RBAC: Authorizing `identity:create_grant(project_id=21b141f7b6394853b423e3366c8e8692, user_id=4905b1221a0d44d99c8f0e739e4a8c94, role_id=de2d2bd26fff4748a79990f97a107164)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:16.938415 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2110333b-9d10-4045-9f70-0257c18dc133 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:16.950856 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bab8df8-6728-4d44-9ad3-44cb68784931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:16.952187 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bab8df8-6728-4d44-9ad3-44cb68784931 admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:16.952766 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bab8df8-6728-4d44-9ad3-44cb68784931 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:16.953257 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bab8df8-6728-4d44-9ad3-44cb68784931 admin admin] PATH_INFO: `/v3/domains` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:16.954847 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bab8df8-6728-4d44-9ad3-44cb68784931 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 12:01:16.955113 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 872/1738] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 12:01:16 2022] PUT /identity/v3/projects/21b141f7b6394853b423e3366c8e8692/users/4905b1221a0d44d99c8f0e739e4a8c94/roles/de2d2bd26fff4748a79990f97a107164 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:01:16.955379 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bab8df8-6728-4d44-9ad3-44cb68784931 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:16.955956 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2110333b-9d10-4045-9f70-0257c18dc133 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:16.959278 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bab8df8-6728-4d44-9ad3-44cb68784931 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:16.976545 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 867/1739] 10.222.0.59 () {58 vars in 1153 bytes} [Tue Oct 11 12:01:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 12:01:16.977561 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bab8df8-6728-4d44-9ad3-44cb68784931 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:16.991490 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bab8df8-6728-4d44-9ad3-44cb68784931 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:17.056783 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2110333b-9d10-4045-9f70-0257c18dc133 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:17.063250 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc905798-445e-420b-8ceb-c8959b55f860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:17.064116 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc905798-445e-420b-8ceb-c8959b55f860 admin admin] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:17.064485 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc905798-445e-420b-8ceb-c8959b55f860 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:17.064848 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc905798-445e-420b-8ceb-c8959b55f860 admin admin] PATH_INFO: `/v3/users` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:17.065589 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc905798-445e-420b-8ceb-c8959b55f860 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:17.068169 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc905798-445e-420b-8ceb-c8959b55f860 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:17.090871 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc29c10d-c6ec-4d70-a4f9-17db72fef53e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:17.091945 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc29c10d-c6ec-4d70-a4f9-17db72fef53e admin admin] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:17.092457 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc29c10d-c6ec-4d70-a4f9-17db72fef53e admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:17.092969 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc29c10d-c6ec-4d70-a4f9-17db72fef53e admin admin] PATH_INFO: `/v3/users` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:17.093646 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 873/1740] 10.222.0.59 () {60 vars in 1144 bytes} [Tue Oct 11 12:01:16 2022] POST /identity/v3/users => generated 292 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:01:17.094360 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc29c10d-c6ec-4d70-a4f9-17db72fef53e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:17.095420 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007de720-f3b2-420f-9c09-73f1bc964050 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:17.095420 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007de720-f3b2-420f-9c09-73f1bc964050 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:17.095716 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007de720-f3b2-420f-9c09-73f1bc964050 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:17.099534 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc29c10d-c6ec-4d70-a4f9-17db72fef53e admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:17.133838 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-007de720-f3b2-420f-9c09-73f1bc964050 None None] MFA Rules not processed for user `4905b1221a0d44d99c8f0e739e4a8c94`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:01:17.135634 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-007de720-f3b2-420f-9c09-73f1bc964050 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:17.140122 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 868/1741] 10.222.0.59 () {60 vars in 1144 bytes} [Tue Oct 11 12:01:16 2022] POST /identity/v3/users => generated 283 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:01:17.143528 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc29c10d-c6ec-4d70-a4f9-17db72fef53e admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:17.160106 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc29c10d-c6ec-4d70-a4f9-17db72fef53e admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:17.167496 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ef6d42-de73-4dec-9935-b9c9687a8be2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:17.169005 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ef6d42-de73-4dec-9935-b9c9687a8be2 admin admin] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:17.169391 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ef6d42-de73-4dec-9935-b9c9687a8be2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:17.169750 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ef6d42-de73-4dec-9935-b9c9687a8be2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:17.170516 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ef6d42-de73-4dec-9935-b9c9687a8be2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:17.173192 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ef6d42-de73-4dec-9935-b9c9687a8be2 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:17.209072 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 869/1742] 10.222.0.59 () {60 vars in 1153 bytes} [Tue Oct 11 12:01:17 2022] POST /identity/v3/projects => generated 375 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:01:17.209882 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ef6d42-de73-4dec-9935-b9c9687a8be2 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:17.222147 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 874/1743] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:01:17 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:01:17.223087 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007de720-f3b2-420f-9c09-73f1bc964050 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:17.235725 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007de720-f3b2-420f-9c09-73f1bc964050 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:17.248243 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:01:17.251087 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:01:17.313211 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ef6d42-de73-4dec-9935-b9c9687a8be2 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:17.319410 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d8c7dd-6dd3-4594-ae76-148056ef73d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:17.319699 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3058551c-7b68-48b0-a437-839eea868a97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:17.320109 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3058551c-7b68-48b0-a437-839eea868a97 admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:17.320377 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d8c7dd-6dd3-4594-ae76-148056ef73d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:17.320560 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3058551c-7b68-48b0-a437-839eea868a97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:17.320722 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3058551c-7b68-48b0-a437-839eea868a97 admin admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:17.320869 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d8c7dd-6dd3-4594-ae76-148056ef73d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:17.321262 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d8c7dd-6dd3-4594-ae76-148056ef73d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:17.321494 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3058551c-7b68-48b0-a437-839eea868a97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:17.321977 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d8c7dd-6dd3-4594-ae76-148056ef73d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:17.323101 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3058551c-7b68-48b0-a437-839eea868a97 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:17.323829 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d8c7dd-6dd3-4594-ae76-148056ef73d1 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:17.337283 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 875/1744] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:01:17 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:01:17.338306 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3058551c-7b68-48b0-a437-839eea868a97 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:17.341198 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 870/1745] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:01:17 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:01:17.342694 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d8c7dd-6dd3-4594-ae76-148056ef73d1 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:17.352675 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3058551c-7b68-48b0-a437-839eea868a97 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:17.356262 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d8c7dd-6dd3-4594-ae76-148056ef73d1 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:17.358719 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d5c8ef-6b32-4bd0-bbb0-7922f0196a91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:17.359365 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d5c8ef-6b32-4bd0-bbb0-7922f0196a91 admin admin] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:17.359644 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d5c8ef-6b32-4bd0-bbb0-7922f0196a91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:17.360113 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d5c8ef-6b32-4bd0-bbb0-7922f0196a91 admin admin] PATH_INFO: `/v3/users` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:17.360786 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d5c8ef-6b32-4bd0-bbb0-7922f0196a91 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:17.363706 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d5c8ef-6b32-4bd0-bbb0-7922f0196a91 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:17.365025 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb11f46-0c4c-465b-88aa-780342b6c41a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:17.366033 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb11f46-0c4c-465b-88aa-780342b6c41a admin admin] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:17.366497 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb11f46-0c4c-465b-88aa-780342b6c41a admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:17.367293 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb11f46-0c4c-465b-88aa-780342b6c41a admin admin] PATH_INFO: `/v3/projects` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:17.368129 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb11f46-0c4c-465b-88aa-780342b6c41a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:17.370665 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb11f46-0c4c-465b-88aa-780342b6c41a admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:17.403856 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 876/1746] 10.222.0.59 () {60 vars in 1144 bytes} [Tue Oct 11 12:01:17 2022] POST /identity/v3/users => generated 288 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:01:17.405653 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d5c8ef-6b32-4bd0-bbb0-7922f0196a91 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:17.423936 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 871/1747] 10.222.0.59 () {60 vars in 1153 bytes} [Tue Oct 11 12:01:17 2022] POST /identity/v3/projects => generated 393 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:01:17.424416 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d5c8ef-6b32-4bd0-bbb0-7922f0196a91 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:17.426141 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb11f46-0c4c-465b-88aa-780342b6c41a admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:17.526306 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb11f46-0c4c-465b-88aa-780342b6c41a admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:17.535329 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120b5d64-dddd-4355-b733-7ee29a7524a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:17.536679 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120b5d64-dddd-4355-b733-7ee29a7524a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:17.537115 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120b5d64-dddd-4355-b733-7ee29a7524a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:17.537581 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120b5d64-dddd-4355-b733-7ee29a7524a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:17.538716 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120b5d64-dddd-4355-b733-7ee29a7524a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:17.541620 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120b5d64-dddd-4355-b733-7ee29a7524a3 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:17.557768 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2818dc62-4913-4e8d-bcd8-e90cfbda974e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:17.559506 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2818dc62-4913-4e8d-bcd8-e90cfbda974e admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:17.560342 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2818dc62-4913-4e8d-bcd8-e90cfbda974e admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:17.560680 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 872/1748] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:01:17 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:01:17.560933 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2818dc62-4913-4e8d-bcd8-e90cfbda974e admin admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:17.561608 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120b5d64-dddd-4355-b733-7ee29a7524a3 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:17.562572 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2818dc62-4913-4e8d-bcd8-e90cfbda974e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:17.566956 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2818dc62-4913-4e8d-bcd8-e90cfbda974e admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:17.585222 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120b5d64-dddd-4355-b733-7ee29a7524a3 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:17.591515 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3638ed-b3cb-4a3c-bd9b-498a25feba99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:17.592163 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3638ed-b3cb-4a3c-bd9b-498a25feba99 admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:17.592608 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3638ed-b3cb-4a3c-bd9b-498a25feba99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:17.592817 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3638ed-b3cb-4a3c-bd9b-498a25feba99 admin admin] PATH_INFO: `/v3/roles` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:17.593410 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3638ed-b3cb-4a3c-bd9b-498a25feba99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:17.595164 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3638ed-b3cb-4a3c-bd9b-498a25feba99 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:17.603925 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 877/1749] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:01:17 2022] GET /identity/v3/roles => generated 1420 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:01:17.605850 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2818dc62-4913-4e8d-bcd8-e90cfbda974e admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:17.614480 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 873/1750] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:01:17 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:01:17.614480 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3638ed-b3cb-4a3c-bd9b-498a25feba99 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:17.626884 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2818dc62-4913-4e8d-bcd8-e90cfbda974e admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:17.636041 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da11c5c7-4634-4c69-afe7-4debf01ce1bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:17.636588 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3638ed-b3cb-4a3c-bd9b-498a25feba99 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:17.637121 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da11c5c7-4634-4c69-afe7-4debf01ce1bd admin admin] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:17.637832 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da11c5c7-4634-4c69-afe7-4debf01ce1bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:17.638323 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da11c5c7-4634-4c69-afe7-4debf01ce1bd admin admin] PATH_INFO: `/v3/users` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:17.639556 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da11c5c7-4634-4c69-afe7-4debf01ce1bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:17.643948 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da11c5c7-4634-4c69-afe7-4debf01ce1bd admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:17.646681 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cd0166-cfad-4305-8365-08dda089ef7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:17.647880 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cd0166-cfad-4305-8365-08dda089ef7b admin admin] REQUEST_METHOD: `PUT` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:17.648440 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cd0166-cfad-4305-8365-08dda089ef7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:17.649017 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cd0166-cfad-4305-8365-08dda089ef7b admin admin] PATH_INFO: `/v3/projects/80b50c7217d740778d256b877daee1d0/users/5b4cbb0e3df44c1aab3bb90ebf6cf8c1/roles/de2d2bd26fff4748a79990f97a107164` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:17.676509 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cd0166-cfad-4305-8365-08dda089ef7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=80b50c7217d740778d256b877daee1d0, user_id=5b4cbb0e3df44c1aab3bb90ebf6cf8c1, role_id=de2d2bd26fff4748a79990f97a107164)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:17.679451 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cd0166-cfad-4305-8365-08dda089ef7b admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:17.679658 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 878/1751] 10.222.0.59 () {60 vars in 1144 bytes} [Tue Oct 11 12:01:17 2022] POST /identity/v3/users => generated 303 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:01:17.680790 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da11c5c7-4634-4c69-afe7-4debf01ce1bd admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:17.695243 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da11c5c7-4634-4c69-afe7-4debf01ce1bd admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:17.700137 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 874/1752] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 12:01:17 2022] PUT /identity/v3/projects/80b50c7217d740778d256b877daee1d0/users/5b4cbb0e3df44c1aab3bb90ebf6cf8c1/roles/de2d2bd26fff4748a79990f97a107164 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:01:17.701702 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cd0166-cfad-4305-8365-08dda089ef7b admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:17.767965 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f29d1a-082d-4ae0-a299-f06c78f67ee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:17.768578 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f29d1a-082d-4ae0-a299-f06c78f67ee4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:17.768847 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f29d1a-082d-4ae0-a299-f06c78f67ee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:17.769117 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f29d1a-082d-4ae0-a299-f06c78f67ee4 admin admin] PATH_INFO: `/v3/projects/0f914c3c6fa84cfaa587ed1a9e9ef1e2/users/f8a601880c6640a0ac0dd6b6299ac4a9/roles/de2d2bd26fff4748a79990f97a107164` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:17.786879 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f29d1a-082d-4ae0-a299-f06c78f67ee4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f914c3c6fa84cfaa587ed1a9e9ef1e2, user_id=f8a601880c6640a0ac0dd6b6299ac4a9, role_id=de2d2bd26fff4748a79990f97a107164)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:17.790135 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f29d1a-082d-4ae0-a299-f06c78f67ee4 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:17.797610 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:01:17.801349 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:01:17.805079 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cd0166-cfad-4305-8365-08dda089ef7b admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:17.809220 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 879/1753] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 12:01:17 2022] PUT /identity/v3/projects/0f914c3c6fa84cfaa587ed1a9e9ef1e2/users/f8a601880c6640a0ac0dd6b6299ac4a9/roles/de2d2bd26fff4748a79990f97a107164 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:01:17.815885 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f29d1a-082d-4ae0-a299-f06c78f67ee4 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:17.907992 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc2e05b-f4b7-43ab-9924-d91a63335a19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:17.908349 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc2e05b-f4b7-43ab-9924-d91a63335a19 admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:17.908682 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc2e05b-f4b7-43ab-9924-d91a63335a19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:17.909331 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc2e05b-f4b7-43ab-9924-d91a63335a19 admin admin] PATH_INFO: `/v3/roles` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:17.910129 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc2e05b-f4b7-43ab-9924-d91a63335a19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:17.913751 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc2e05b-f4b7-43ab-9924-d91a63335a19 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:17.945093 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f29d1a-082d-4ae0-a299-f06c78f67ee4 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:17.951631 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fc1aee-97a2-48df-be30-2a60934b8fb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:17.952327 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 875/1754] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:01:17 2022] GET /identity/v3/roles => generated 1420 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:01:17.953162 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fc1aee-97a2-48df-be30-2a60934b8fb8 admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:17.953636 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fc1aee-97a2-48df-be30-2a60934b8fb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:17.954325 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fc1aee-97a2-48df-be30-2a60934b8fb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:17.954949 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028038da-b01e-4a2b-b83a-b088ad6f371b None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:17.955600 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028038da-b01e-4a2b-b83a-b088ad6f371b None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:17.955752 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fc1aee-97a2-48df-be30-2a60934b8fb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:17.956120 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028038da-b01e-4a2b-b83a-b088ad6f371b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:17.958091 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fc1aee-97a2-48df-be30-2a60934b8fb8 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:17.981761 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 880/1755] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:01:17 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:01:17.983053 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f82ea08-34c8-4e17-a324-843da7e11d69 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:17.984211 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f82ea08-34c8-4e17-a324-843da7e11d69 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:17.984211 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f82ea08-34c8-4e17-a324-843da7e11d69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:18.009949 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-028038da-b01e-4a2b-b83a-b088ad6f371b None None] MFA Rules not processed for user `5b4cbb0e3df44c1aab3bb90ebf6cf8c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:01:18.011833 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-028038da-b01e-4a2b-b83a-b088ad6f371b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:18.023259 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-6f82ea08-34c8-4e17-a324-843da7e11d69 None None] MFA Rules not processed for user `f8a601880c6640a0ac0dd6b6299ac4a9`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:01:18.024492 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-6f82ea08-34c8-4e17-a324-843da7e11d69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:18.127324 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 876/1756] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:01:17 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:01:18.127324 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028038da-b01e-4a2b-b83a-b088ad6f371b None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:18.137348 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 881/1757] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:01:17 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:01:18.138256 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f82ea08-34c8-4e17-a324-843da7e11d69 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:18.141434 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028038da-b01e-4a2b-b83a-b088ad6f371b None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:18.147751 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73445faa-5b09-439b-8047-39d87766a49b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:18.148609 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73445faa-5b09-439b-8047-39d87766a49b admin admin] REQUEST_METHOD: `PUT` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:18.148996 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73445faa-5b09-439b-8047-39d87766a49b admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:18.149387 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73445faa-5b09-439b-8047-39d87766a49b admin admin] PATH_INFO: `/v3/projects/b416a9fa6f7644cab2f7353f91d15ef7/users/60433b5cf90c4fdc8ded12fb2020e4c7/roles/de2d2bd26fff4748a79990f97a107164` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:18.150448 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f82ea08-34c8-4e17-a324-843da7e11d69 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:18.156639 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e9a50b-a3fe-4696-a320-9e7bdc5c0da1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:18.157393 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e9a50b-a3fe-4696-a320-9e7bdc5c0da1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:18.157773 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e9a50b-a3fe-4696-a320-9e7bdc5c0da1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:18.158043 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e9a50b-a3fe-4696-a320-9e7bdc5c0da1 admin admin] PATH_INFO: `/v3/projects/af431dae2476490196b40d063e704a84/users/d7f8df19cc5941688031c3bee68917d0/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:18.175331 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e9a50b-a3fe-4696-a320-9e7bdc5c0da1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af431dae2476490196b40d063e704a84, user_id=d7f8df19cc5941688031c3bee68917d0, role_id=5cf1f4bd2fef46ab9c6eff56d52f9f69)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:18.178927 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e9a50b-a3fe-4696-a320-9e7bdc5c0da1 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:18.179926 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73445faa-5b09-439b-8047-39d87766a49b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b416a9fa6f7644cab2f7353f91d15ef7, user_id=60433b5cf90c4fdc8ded12fb2020e4c7, role_id=de2d2bd26fff4748a79990f97a107164)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:18.185379 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73445faa-5b09-439b-8047-39d87766a49b admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:18.197651 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 882/1758] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 12:01:18 2022] PUT /identity/v3/projects/af431dae2476490196b40d063e704a84/users/d7f8df19cc5941688031c3bee68917d0/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:01:18.201240 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e9a50b-a3fe-4696-a320-9e7bdc5c0da1 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:18.210805 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 877/1759] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 12:01:18 2022] PUT /identity/v3/projects/b416a9fa6f7644cab2f7353f91d15ef7/users/60433b5cf90c4fdc8ded12fb2020e4c7/roles/de2d2bd26fff4748a79990f97a107164 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:01:18.212402 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73445faa-5b09-439b-8047-39d87766a49b admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:18.289810 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e9a50b-a3fe-4696-a320-9e7bdc5c0da1 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:18.296485 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:01:18.306999 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73445faa-5b09-439b-8047-39d87766a49b admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:18.312734 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b62289-cdf8-40a5-ae3a-5707804c9fd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:18.313417 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b62289-cdf8-40a5-ae3a-5707804c9fd8 admin admin] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:18.313772 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b62289-cdf8-40a5-ae3a-5707804c9fd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:18.314027 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b62289-cdf8-40a5-ae3a-5707804c9fd8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:18.314744 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b62289-cdf8-40a5-ae3a-5707804c9fd8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:18.316740 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b62289-cdf8-40a5-ae3a-5707804c9fd8 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:18.330855 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:01:18.359123 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 878/1760] 10.222.0.59 () {60 vars in 1153 bytes} [Tue Oct 11 12:01:18 2022] POST /identity/v3/projects => generated 367 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:01:18.360417 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b62289-cdf8-40a5-ae3a-5707804c9fd8 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:18.394598 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f4eadf-f968-47c9-b030-52874ba22ca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:18.395590 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f4eadf-f968-47c9-b030-52874ba22ca7 admin admin] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:18.396050 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f4eadf-f968-47c9-b030-52874ba22ca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:18.396469 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f4eadf-f968-47c9-b030-52874ba22ca7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:18.397269 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f4eadf-f968-47c9-b030-52874ba22ca7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:18.399580 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f4eadf-f968-47c9-b030-52874ba22ca7 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:18.440126 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 883/1761] 10.222.0.59 () {60 vars in 1153 bytes} [Tue Oct 11 12:01:18 2022] POST /identity/v3/projects => generated 385 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:01:18.442315 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6461aa40-7f98-4868-9164-c96e59bcd339 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:18.442809 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6461aa40-7f98-4868-9164-c96e59bcd339 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:18.443423 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6461aa40-7f98-4868-9164-c96e59bcd339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:18.466003 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b62289-cdf8-40a5-ae3a-5707804c9fd8 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:18.483709 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-6461aa40-7f98-4868-9164-c96e59bcd339 None None] MFA Rules not processed for user `60433b5cf90c4fdc8ded12fb2020e4c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:01:18.487743 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-6461aa40-7f98-4868-9164-c96e59bcd339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:18.567154 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311d51d7-2359-4e2a-9fe7-75e325029492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:18.568108 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311d51d7-2359-4e2a-9fe7-75e325029492 admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:18.568533 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311d51d7-2359-4e2a-9fe7-75e325029492 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:18.568943 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311d51d7-2359-4e2a-9fe7-75e325029492 admin admin] PATH_INFO: `/v3/roles` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:18.569991 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311d51d7-2359-4e2a-9fe7-75e325029492 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:18.572947 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311d51d7-2359-4e2a-9fe7-75e325029492 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:18.577837 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 884/1762] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:01:18 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:01:18.578836 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6461aa40-7f98-4868-9164-c96e59bcd339 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:18.596737 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 879/1763] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:01:18 2022] GET /identity/v3/roles => generated 1420 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:01:18.596737 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311d51d7-2359-4e2a-9fe7-75e325029492 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:18.597088 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6461aa40-7f98-4868-9164-c96e59bcd339 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:18.600160 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f5afaf-1731-46db-9747-b6205b9d88a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:18.600850 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f5afaf-1731-46db-9747-b6205b9d88a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:18.601129 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f5afaf-1731-46db-9747-b6205b9d88a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:18.601392 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f5afaf-1731-46db-9747-b6205b9d88a9 admin admin] PATH_INFO: `/v3/users` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:18.602126 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f5afaf-1731-46db-9747-b6205b9d88a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:18.604201 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f5afaf-1731-46db-9747-b6205b9d88a9 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:18.612035 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311d51d7-2359-4e2a-9fe7-75e325029492 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:18.620132 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e621be2-0620-47c1-b41b-7605dfa2fbb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:18.621209 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e621be2-0620-47c1-b41b-7605dfa2fbb9 admin admin] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:18.621700 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e621be2-0620-47c1-b41b-7605dfa2fbb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:18.622175 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e621be2-0620-47c1-b41b-7605dfa2fbb9 admin admin] PATH_INFO: `/v3/users` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:18.623411 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e621be2-0620-47c1-b41b-7605dfa2fbb9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:18.627360 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e621be2-0620-47c1-b41b-7605dfa2fbb9 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:18.635015 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 885/1764] 10.222.0.59 () {60 vars in 1144 bytes} [Tue Oct 11 12:01:18 2022] POST /identity/v3/users => generated 290 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:01:18.635015 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f5afaf-1731-46db-9747-b6205b9d88a9 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:18.647575 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f5afaf-1731-46db-9747-b6205b9d88a9 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:18.653979 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1e2348-3bc4-4c9a-8e78-1844f382c728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:18.655044 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1e2348-3bc4-4c9a-8e78-1844f382c728 admin admin] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:18.655326 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1e2348-3bc4-4c9a-8e78-1844f382c728 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:18.655621 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1e2348-3bc4-4c9a-8e78-1844f382c728 admin admin] PATH_INFO: `/v3/projects` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:18.656281 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1e2348-3bc4-4c9a-8e78-1844f382c728 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:18.658432 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1e2348-3bc4-4c9a-8e78-1844f382c728 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:18.663576 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 880/1765] 10.222.0.59 () {60 vars in 1144 bytes} [Tue Oct 11 12:01:18 2022] POST /identity/v3/users => generated 299 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:01:18.665116 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e621be2-0620-47c1-b41b-7605dfa2fbb9 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:18.680495 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e621be2-0620-47c1-b41b-7605dfa2fbb9 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:18.686709 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fec4b81-e86f-45d7-88cb-0b20b0d0ed93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:18.687514 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fec4b81-e86f-45d7-88cb-0b20b0d0ed93 admin admin] REQUEST_METHOD: `PUT` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:18.687880 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fec4b81-e86f-45d7-88cb-0b20b0d0ed93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:18.688154 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fec4b81-e86f-45d7-88cb-0b20b0d0ed93 admin admin] PATH_INFO: `/v3/projects/af431dae2476490196b40d063e704a84/users/d7f8df19cc5941688031c3bee68917d0/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:18.690918 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fec4b81-e86f-45d7-88cb-0b20b0d0ed93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af431dae2476490196b40d063e704a84, user_id=d7f8df19cc5941688031c3bee68917d0, role_id=5cf1f4bd2fef46ab9c6eff56d52f9f69)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:18.694047 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fec4b81-e86f-45d7-88cb-0b20b0d0ed93 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:18.703562 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 886/1766] 10.222.0.59 () {60 vars in 1153 bytes} [Tue Oct 11 12:01:18 2022] POST /identity/v3/projects => generated 375 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:01:18.704925 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1e2348-3bc4-4c9a-8e78-1844f382c728 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:18.719709 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 881/1767] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 12:01:18 2022] PUT /identity/v3/projects/af431dae2476490196b40d063e704a84/users/d7f8df19cc5941688031c3bee68917d0/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:01:18.719709 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fec4b81-e86f-45d7-88cb-0b20b0d0ed93 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:18.791256 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:01:18.817833 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1e2348-3bc4-4c9a-8e78-1844f382c728 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:18.823326 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2082f5-0bbb-4627-88d3-52c189f2e418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:18.823984 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2082f5-0bbb-4627-88d3-52c189f2e418 admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:18.824263 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2082f5-0bbb-4627-88d3-52c189f2e418 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:18.829083 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fec4b81-e86f-45d7-88cb-0b20b0d0ed93 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:18.832208 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2082f5-0bbb-4627-88d3-52c189f2e418 admin admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:18.833529 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2082f5-0bbb-4627-88d3-52c189f2e418 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:18.835337 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2082f5-0bbb-4627-88d3-52c189f2e418 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:18.837921 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f137afc-51bb-456e-b79a-d93c437a1def admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:18.839562 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f137afc-51bb-456e-b79a-d93c437a1def admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:18.840305 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f137afc-51bb-456e-b79a-d93c437a1def admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:18.841019 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f137afc-51bb-456e-b79a-d93c437a1def admin admin] PATH_INFO: `/v3/roles` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:18.842670 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f137afc-51bb-456e-b79a-d93c437a1def admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:18.846598 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f137afc-51bb-456e-b79a-d93c437a1def admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:18.847256 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 887/1768] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:01:18 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:01:18.848399 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2082f5-0bbb-4627-88d3-52c189f2e418 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:18.858411 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:01:18.861940 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2082f5-0bbb-4627-88d3-52c189f2e418 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:18.867492 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2fd48e-1573-4512-a69b-3d7f58baabd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:18.868138 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2fd48e-1573-4512-a69b-3d7f58baabd2 admin admin] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:18.868417 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2fd48e-1573-4512-a69b-3d7f58baabd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:18.868720 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2fd48e-1573-4512-a69b-3d7f58baabd2 admin admin] PATH_INFO: `/v3/users` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:18.869374 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2fd48e-1573-4512-a69b-3d7f58baabd2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:18.871430 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2fd48e-1573-4512-a69b-3d7f58baabd2 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:18.874099 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 882/1769] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:01:18 2022] GET /identity/v3/roles => generated 1420 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:01:18.876888 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a48992e-cf1a-4de5-85b4-667a071f055c None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:18.877363 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a48992e-cf1a-4de5-85b4-667a071f055c None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:18.878112 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a48992e-cf1a-4de5-85b4-667a071f055c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:18.903671 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 888/1770] 10.222.0.59 () {60 vars in 1144 bytes} [Tue Oct 11 12:01:18 2022] POST /identity/v3/users => generated 294 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:01:18.903671 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2fd48e-1573-4512-a69b-3d7f58baabd2 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:18.924933 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2fd48e-1573-4512-a69b-3d7f58baabd2 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:18.930115 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d65225-00ad-46ba-9f91-19ecf36dcb65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:18.931307 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d65225-00ad-46ba-9f91-19ecf36dcb65 admin admin] REQUEST_METHOD: `PUT` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:18.931421 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-7a48992e-cf1a-4de5-85b4-667a071f055c None None] MFA Rules not processed for user `d7f8df19cc5941688031c3bee68917d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:01:18.931688 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d65225-00ad-46ba-9f91-19ecf36dcb65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:18.931981 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d65225-00ad-46ba-9f91-19ecf36dcb65 admin admin] PATH_INFO: `/v3/projects/45f5dd0d6f4044689b2babcb323d2b5b/users/17dbccf6199140a3839fc6e176643648/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:18.933030 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-7a48992e-cf1a-4de5-85b4-667a071f055c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:18.953787 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d65225-00ad-46ba-9f91-19ecf36dcb65 admin admin] RBAC: Authorizing `identity:create_grant(project_id=45f5dd0d6f4044689b2babcb323d2b5b, user_id=17dbccf6199140a3839fc6e176643648, role_id=5cf1f4bd2fef46ab9c6eff56d52f9f69)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:18.957007 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d65225-00ad-46ba-9f91-19ecf36dcb65 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:18.977637 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 889/1771] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 12:01:18 2022] PUT /identity/v3/projects/45f5dd0d6f4044689b2babcb323d2b5b/users/17dbccf6199140a3839fc6e176643648/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:01:18.978842 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d65225-00ad-46ba-9f91-19ecf36dcb65 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:19.027351 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 883/1772] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:01:18 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:01:19.028260 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a48992e-cf1a-4de5-85b4-667a071f055c None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:19.039454 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a48992e-cf1a-4de5-85b4-667a071f055c None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:19.059335 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d65225-00ad-46ba-9f91-19ecf36dcb65 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:19.064461 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1430375-decb-4bdb-9545-68b72bd78bdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:19.065164 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1430375-decb-4bdb-9545-68b72bd78bdb admin admin] REQUEST_METHOD: `PUT` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:19.065450 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1430375-decb-4bdb-9545-68b72bd78bdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:19.065713 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1430375-decb-4bdb-9545-68b72bd78bdb admin admin] PATH_INFO: `/v3/projects/d8214d1e985a4188b701f104e45f6805/users/c476c7c73fde4fdca1a27f0e0bbc3499/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:19.082155 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1430375-decb-4bdb-9545-68b72bd78bdb admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8214d1e985a4188b701f104e45f6805, user_id=c476c7c73fde4fdca1a27f0e0bbc3499, role_id=5cf1f4bd2fef46ab9c6eff56d52f9f69)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:19.085022 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1430375-decb-4bdb-9545-68b72bd78bdb admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:19.102910 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 890/1773] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 12:01:18 2022] PUT /identity/v3/projects/d8214d1e985a4188b701f104e45f6805/users/c476c7c73fde4fdca1a27f0e0bbc3499/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:01:19.104309 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1430375-decb-4bdb-9545-68b72bd78bdb admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:19.105057 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4795f09-3eae-4ff2-b241-b9d1e2fa8b84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:19.105855 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4795f09-3eae-4ff2-b241-b9d1e2fa8b84 admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:19.106206 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4795f09-3eae-4ff2-b241-b9d1e2fa8b84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:19.106548 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4795f09-3eae-4ff2-b241-b9d1e2fa8b84 admin admin] PATH_INFO: `/v3/roles` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:19.107331 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4795f09-3eae-4ff2-b241-b9d1e2fa8b84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:19.109052 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4795f09-3eae-4ff2-b241-b9d1e2fa8b84 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:19.127168 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 884/1774] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:01:19 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:01:19.127168 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4795f09-3eae-4ff2-b241-b9d1e2fa8b84 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:19.127168 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-a4795f09-3eae-4ff2-b241-b9d1e2fa8b84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:19.180690 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1430375-decb-4bdb-9545-68b72bd78bdb admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:19.185682 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbf6618-3fdc-4345-88c7-d917af9d7098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:19.186325 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbf6618-3fdc-4345-88c7-d917af9d7098 admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:19.186628 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbf6618-3fdc-4345-88c7-d917af9d7098 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:19.186906 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbf6618-3fdc-4345-88c7-d917af9d7098 admin admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:19.187576 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbf6618-3fdc-4345-88c7-d917af9d7098 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:19.190405 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbf6618-3fdc-4345-88c7-d917af9d7098 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:19.196891 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4795f09-3eae-4ff2-b241-b9d1e2fa8b84 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:19.204730 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 891/1775] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:01:19 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:01:19.204730 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbf6618-3fdc-4345-88c7-d917af9d7098 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:19.212589 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbf6618-3fdc-4345-88c7-d917af9d7098 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:19.217520 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6697b71c-59e3-4b1a-8d1b-9887003b0e41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:19.218156 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6697b71c-59e3-4b1a-8d1b-9887003b0e41 admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:19.218445 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6697b71c-59e3-4b1a-8d1b-9887003b0e41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:19.218760 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6697b71c-59e3-4b1a-8d1b-9887003b0e41 admin admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:19.219418 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6697b71c-59e3-4b1a-8d1b-9887003b0e41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:19.221218 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6697b71c-59e3-4b1a-8d1b-9887003b0e41 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:19.235504 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 892/1776] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:01:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:01:19.235504 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6697b71c-59e3-4b1a-8d1b-9887003b0e41 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:19.239860 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:01:19.244410 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6697b71c-59e3-4b1a-8d1b-9887003b0e41 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:19.249632 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:01:19.251861 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfa17f3-9520-4749-b3a5-90c518824782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:19.252570 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfa17f3-9520-4749-b3a5-90c518824782 admin admin] REQUEST_METHOD: `PUT` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:19.253173 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfa17f3-9520-4749-b3a5-90c518824782 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:19.253773 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfa17f3-9520-4749-b3a5-90c518824782 admin admin] PATH_INFO: `/v3/projects/9a54b2b3fce8405080248bbb889c67bf/users/f5fa55a8a7a247959550948f115f866a/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:19.269145 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a549cfc4-5790-4fb1-8d9b-e3311cb846ae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b1b651c73664d08accc6edf2f79bace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:19.270179 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a549cfc4-5790-4fb1-8d9b-e3311cb846ae service cinder] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:19.270616 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a549cfc4-5790-4fb1-8d9b-e3311cb846ae service cinder] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:19.271026 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a549cfc4-5790-4fb1-8d9b-e3311cb846ae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:19.272262 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfa17f3-9520-4749-b3a5-90c518824782 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a54b2b3fce8405080248bbb889c67bf, user_id=f5fa55a8a7a247959550948f115f866a, role_id=5cf1f4bd2fef46ab9c6eff56d52f9f69)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:19.275110 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfa17f3-9520-4749-b3a5-90c518824782 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:19.290289 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 893/1777] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 12:01:19 2022] PUT /identity/v3/projects/9a54b2b3fce8405080248bbb889c67bf/users/f5fa55a8a7a247959550948f115f866a/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:01:19.291199 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfa17f3-9520-4749-b3a5-90c518824782 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:19.349729 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a549cfc4-5790-4fb1-8d9b-e3311cb846ae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:19.351370 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a549cfc4-5790-4fb1-8d9b-e3311cb846ae service cinder] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:19.363671 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 885/1778] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:01:19 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:01:19.365127 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a549cfc4-5790-4fb1-8d9b-e3311cb846ae service cinder] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:19.370138 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfa17f3-9520-4749-b3a5-90c518824782 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:19.377955 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae5c257-ae78-4b55-8ccd-94b2d9f2e170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:19.378768 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae5c257-ae78-4b55-8ccd-94b2d9f2e170 admin admin] REQUEST_METHOD: `PUT` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:19.379119 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae5c257-ae78-4b55-8ccd-94b2d9f2e170 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:19.379323 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae5c257-ae78-4b55-8ccd-94b2d9f2e170 admin admin] PATH_INFO: `/v3/projects/45f5dd0d6f4044689b2babcb323d2b5b/users/17dbccf6199140a3839fc6e176643648/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:19.381534 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae5c257-ae78-4b55-8ccd-94b2d9f2e170 admin admin] RBAC: Authorizing `identity:create_grant(project_id=45f5dd0d6f4044689b2babcb323d2b5b, user_id=17dbccf6199140a3839fc6e176643648, role_id=5cf1f4bd2fef46ab9c6eff56d52f9f69)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:19.383146 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a549cfc4-5790-4fb1-8d9b-e3311cb846ae service cinder] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:19.385232 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae5c257-ae78-4b55-8ccd-94b2d9f2e170 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:19.389647 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174e551d-dc6c-44ce-b3df-71d3b8c3f8a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:19.390705 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174e551d-dc6c-44ce-b3df-71d3b8c3f8a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:19.391146 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174e551d-dc6c-44ce-b3df-71d3b8c3f8a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:19.392201 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174e551d-dc6c-44ce-b3df-71d3b8c3f8a9 admin admin] PATH_INFO: `/v3/projects/d8214d1e985a4188b701f104e45f6805/users/c476c7c73fde4fdca1a27f0e0bbc3499/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:19.395307 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174e551d-dc6c-44ce-b3df-71d3b8c3f8a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8214d1e985a4188b701f104e45f6805, user_id=c476c7c73fde4fdca1a27f0e0bbc3499, role_id=5cf1f4bd2fef46ab9c6eff56d52f9f69)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:19.398148 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 894/1779] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 12:01:19 2022] PUT /identity/v3/projects/45f5dd0d6f4044689b2babcb323d2b5b/users/17dbccf6199140a3839fc6e176643648/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:01:19.398943 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae5c257-ae78-4b55-8ccd-94b2d9f2e170 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:19.399199 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174e551d-dc6c-44ce-b3df-71d3b8c3f8a9 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:19.427641 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 886/1780] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 12:01:19 2022] PUT /identity/v3/projects/d8214d1e985a4188b701f104e45f6805/users/c476c7c73fde4fdca1a27f0e0bbc3499/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:01:19.430260 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55ae4ed-3ff4-430d-b61c-c46b44579991 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:19.430781 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55ae4ed-3ff4-430d-b61c-c46b44579991 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:19.431387 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55ae4ed-3ff4-430d-b61c-c46b44579991 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:19.468486 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-a55ae4ed-3ff4-430d-b61c-c46b44579991 None None] MFA Rules not processed for user `17dbccf6199140a3839fc6e176643648`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:01:19.469720 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-a55ae4ed-3ff4-430d-b61c-c46b44579991 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:19.482500 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae5c257-ae78-4b55-8ccd-94b2d9f2e170 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:19.562370 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 887/1781] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:01:19 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:01:19.562370 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55ae4ed-3ff4-430d-b61c-c46b44579991 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:19.562370 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-a55ae4ed-3ff4-430d-b61c-c46b44579991 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:19.583752 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b49f162-d702-45cb-8c1a-640b6197d28f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:19.584681 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b49f162-d702-45cb-8c1a-640b6197d28f admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:19.584987 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b49f162-d702-45cb-8c1a-640b6197d28f admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:19.585483 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b49f162-d702-45cb-8c1a-640b6197d28f admin admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:19.586513 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b49f162-d702-45cb-8c1a-640b6197d28f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:19.589786 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b49f162-d702-45cb-8c1a-640b6197d28f admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:19.604648 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 895/1782] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:01:19 2022] GET /identity/v3/roles => generated 1420 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:01:19.605960 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78015366-4c05-497f-892a-a1224ece37e9 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:19.606249 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78015366-4c05-497f-892a-a1224ece37e9 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:19.606598 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78015366-4c05-497f-892a-a1224ece37e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:19.642433 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-78015366-4c05-497f-892a-a1224ece37e9 None None] MFA Rules not processed for user `c476c7c73fde4fdca1a27f0e0bbc3499`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:01:19.644031 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-78015366-4c05-497f-892a-a1224ece37e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:19.650669 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:01:19.651120 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55ae4ed-3ff4-430d-b61c-c46b44579991 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:19.655692 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd9dca7-6b6e-4756-9be2-b7668aa68756 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b1b651c73664d08accc6edf2f79bace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:19.656322 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd9dca7-6b6e-4756-9be2-b7668aa68756 service cinder] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:19.656611 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd9dca7-6b6e-4756-9be2-b7668aa68756 service cinder] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:19.657046 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd9dca7-6b6e-4756-9be2-b7668aa68756 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:19.672496 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:01:19.727452 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd9dca7-6b6e-4756-9be2-b7668aa68756 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:19.727860 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 896/1783] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:01:19 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:01:19.728905 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78015366-4c05-497f-892a-a1224ece37e9 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:19.729063 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd9dca7-6b6e-4756-9be2-b7668aa68756 service cinder] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:19.740610 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78015366-4c05-497f-892a-a1224ece37e9 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:19.741713 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 888/1784] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:01:19 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:01:19.742653 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd9dca7-6b6e-4756-9be2-b7668aa68756 service cinder] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:19.744946 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d5356b-7809-4cd1-baaf-cbb1b724cf64 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b1b651c73664d08accc6edf2f79bace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:19.745585 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d5356b-7809-4cd1-baaf-cbb1b724cf64 service cinder] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:19.745865 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d5356b-7809-4cd1-baaf-cbb1b724cf64 service cinder] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:19.746451 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d5356b-7809-4cd1-baaf-cbb1b724cf64 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:19.755525 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd9dca7-6b6e-4756-9be2-b7668aa68756 service cinder] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:19.757285 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d5356b-7809-4cd1-baaf-cbb1b724cf64 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:19.758772 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d5356b-7809-4cd1-baaf-cbb1b724cf64 service cinder] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:19.760930 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f58e36d-5275-4b1e-af09-1b02c405301c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:19.761621 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f58e36d-5275-4b1e-af09-1b02c405301c admin admin] REQUEST_METHOD: `PUT` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:19.761900 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f58e36d-5275-4b1e-af09-1b02c405301c admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:19.762173 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f58e36d-5275-4b1e-af09-1b02c405301c admin admin] PATH_INFO: `/v3/projects/9a54b2b3fce8405080248bbb889c67bf/users/f5fa55a8a7a247959550948f115f866a/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:19.764990 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f58e36d-5275-4b1e-af09-1b02c405301c admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a54b2b3fce8405080248bbb889c67bf, user_id=f5fa55a8a7a247959550948f115f866a, role_id=5cf1f4bd2fef46ab9c6eff56d52f9f69)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:19.767620 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f58e36d-5275-4b1e-af09-1b02c405301c admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:19.770690 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 897/1785] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:01:19 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:01:19.780638 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 889/1786] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 12:01:19 2022] PUT /identity/v3/projects/9a54b2b3fce8405080248bbb889c67bf/users/f5fa55a8a7a247959550948f115f866a/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:01:19.795444 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27cf971-f193-4a86-9a12-7d826ace5446 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:19.795913 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27cf971-f193-4a86-9a12-7d826ace5446 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:19.796447 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27cf971-f193-4a86-9a12-7d826ace5446 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:19.839231 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-f27cf971-f193-4a86-9a12-7d826ace5446 None None] MFA Rules not processed for user `f5fa55a8a7a247959550948f115f866a`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:01:19.846661 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-f27cf971-f193-4a86-9a12-7d826ace5446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:19.856177 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f58e36d-5275-4b1e-af09-1b02c405301c admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:19.940229 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f58e36d-5275-4b1e-af09-1b02c405301c admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:19.945225 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56561f1-3600-48a1-a657-6d5b95bd3848 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b1b651c73664d08accc6edf2f79bace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:19.946003 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56561f1-3600-48a1-a657-6d5b95bd3848 service cinder] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:19.946344 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56561f1-3600-48a1-a657-6d5b95bd3848 service cinder] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:19.946710 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56561f1-3600-48a1-a657-6d5b95bd3848 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:19.954244 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 898/1787] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:01:19 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:01:20.051306 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56561f1-3600-48a1-a657-6d5b95bd3848 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:20.054322 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56561f1-3600-48a1-a657-6d5b95bd3848 service cinder] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:20.068469 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 890/1788] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:01:19 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 11 12:01:20.201876 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27cf971-f193-4a86-9a12-7d826ace5446 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:20.223070 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27cf971-f193-4a86-9a12-7d826ace5446 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:20.230313 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95246bee-8c2e-44d1-b622-8f8b0a0898a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b1b651c73664d08accc6edf2f79bace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:20.231705 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95246bee-8c2e-44d1-b622-8f8b0a0898a8 service cinder] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:20.232398 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95246bee-8c2e-44d1-b622-8f8b0a0898a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:20.233059 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95246bee-8c2e-44d1-b622-8f8b0a0898a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:20.334775 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95246bee-8c2e-44d1-b622-8f8b0a0898a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:20.338038 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95246bee-8c2e-44d1-b622-8f8b0a0898a8 service cinder] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:20.352996 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 899/1789] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:01:20 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:01:20.510006 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56561f1-3600-48a1-a657-6d5b95bd3848 service cinder] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:20.531354 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56561f1-3600-48a1-a657-6d5b95bd3848 service cinder] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:20.537483 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379ecb66-d852-462a-a635-9a7982dc36a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b1b651c73664d08accc6edf2f79bace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:20.538449 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379ecb66-d852-462a-a635-9a7982dc36a0 service cinder] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:20.539022 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379ecb66-d852-462a-a635-9a7982dc36a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:20.539737 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379ecb66-d852-462a-a635-9a7982dc36a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:20.616106 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:01:20.624230 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379ecb66-d852-462a-a635-9a7982dc36a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:20.627305 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379ecb66-d852-462a-a635-9a7982dc36a0 service cinder] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:20.643800 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 891/1790] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:01:20 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:01:26.268308 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95246bee-8c2e-44d1-b622-8f8b0a0898a8 service cinder] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:26.279278 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95246bee-8c2e-44d1-b622-8f8b0a0898a8 service cinder] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:26.283307 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6911c3-c0a8-45f6-ac22-559fe9921063 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b1b651c73664d08accc6edf2f79bace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:26.283891 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6911c3-c0a8-45f6-ac22-559fe9921063 service cinder] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:26.284180 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6911c3-c0a8-45f6-ac22-559fe9921063 service cinder] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:26.284536 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6911c3-c0a8-45f6-ac22-559fe9921063 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:26.295604 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:01:26.360816 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6911c3-c0a8-45f6-ac22-559fe9921063 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:26.362434 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6911c3-c0a8-45f6-ac22-559fe9921063 service cinder] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:26.375397 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 900/1791] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:01:26 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:01:27.153994 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379ecb66-d852-462a-a635-9a7982dc36a0 service cinder] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:27.227614 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6911c3-c0a8-45f6-ac22-559fe9921063 service cinder] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:27.293658 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379ecb66-d852-462a-a635-9a7982dc36a0 service cinder] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:27.300691 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9c0463-19b2-4896-bdc1-a97d26cb0c81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:27.301948 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9c0463-19b2-4896-bdc1-a97d26cb0c81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:27.302508 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9c0463-19b2-4896-bdc1-a97d26cb0c81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:27.302985 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9c0463-19b2-4896-bdc1-a97d26cb0c81 admin admin] PATH_INFO: `/v3/users/4905b1221a0d44d99c8f0e739e4a8c94` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:27.304611 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9c0463-19b2-4896-bdc1-a97d26cb0c81 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4905b1221a0d44d99c8f0e739e4a8c94)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:27.307394 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9c0463-19b2-4896-bdc1-a97d26cb0c81 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:27.328286 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6911c3-c0a8-45f6-ac22-559fe9921063 service cinder] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:27.339130 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0206741c-55b3-4812-9361-6e4e729a3adb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:27.340102 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0206741c-55b3-4812-9361-6e4e729a3adb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:27.340448 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0206741c-55b3-4812-9361-6e4e729a3adb admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:27.340791 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0206741c-55b3-4812-9361-6e4e729a3adb admin admin] PATH_INFO: `/v3/users/60433b5cf90c4fdc8ded12fb2020e4c7` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:27.357318 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0206741c-55b3-4812-9361-6e4e729a3adb admin admin] RBAC: Authorizing `identity:delete_user(user_id=60433b5cf90c4fdc8ded12fb2020e4c7)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:27.360440 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0206741c-55b3-4812-9361-6e4e729a3adb admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:27.394273 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-ae9c0463-19b2-4896-bdc1-a97d26cb0c81 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4905b1221a0d44d99c8f0e739e4a8c94'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:27.419730 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-ae9c0463-19b2-4896-bdc1-a97d26cb0c81 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4905b1221a0d44d99c8f0e739e4a8c94'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:27.433097 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-ae9c0463-19b2-4896-bdc1-a97d26cb0c81 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4905b1221a0d44d99c8f0e739e4a8c94'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:27.434073 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-ae9c0463-19b2-4896-bdc1-a97d26cb0c81 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4905b1221a0d44d99c8f0e739e4a8c94'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:27.445647 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-ae9c0463-19b2-4896-bdc1-a97d26cb0c81 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4905b1221a0d44d99c8f0e739e4a8c94'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:27.458202 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 892/1792] 10.222.0.59 () {58 vars in 1224 bytes} [Tue Oct 11 12:01:27 2022] DELETE /identity/v3/users/4905b1221a0d44d99c8f0e739e4a8c94 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:01:27.463635 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570e5c2e-83c4-4afe-9d56-c91f9a67baa6 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:27.464176 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570e5c2e-83c4-4afe-9d56-c91f9a67baa6 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:27.464595 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570e5c2e-83c4-4afe-9d56-c91f9a67baa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:27.536200 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-570e5c2e-83c4-4afe-9d56-c91f9a67baa6 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:01:27.538976 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-0206741c-55b3-4812-9361-6e4e729a3adb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '60433b5cf90c4fdc8ded12fb2020e4c7'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:27.541683 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-570e5c2e-83c4-4afe-9d56-c91f9a67baa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:27.619766 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-0206741c-55b3-4812-9361-6e4e729a3adb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '60433b5cf90c4fdc8ded12fb2020e4c7'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:27.620881 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-0206741c-55b3-4812-9361-6e4e729a3adb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '60433b5cf90c4fdc8ded12fb2020e4c7'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:27.632910 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-0206741c-55b3-4812-9361-6e4e729a3adb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '60433b5cf90c4fdc8ded12fb2020e4c7'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:27.634009 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-0206741c-55b3-4812-9361-6e4e729a3adb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '60433b5cf90c4fdc8ded12fb2020e4c7'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:27.653822 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 901/1793] 10.222.0.59 () {58 vars in 1224 bytes} [Tue Oct 11 12:01:27 2022] DELETE /identity/v3/users/60433b5cf90c4fdc8ded12fb2020e4c7 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:01:27.654959 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0206741c-55b3-4812-9361-6e4e729a3adb admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:27.656854 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-0206741c-55b3-4812-9361-6e4e729a3adb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:27.676011 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 893/1794] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:01:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 12:01:27.678366 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523971c1-ee23-4d59-a8d1-6e9d939aae50 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:27.678878 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523971c1-ee23-4d59-a8d1-6e9d939aae50 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:27.679359 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523971c1-ee23-4d59-a8d1-6e9d939aae50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:27.700705 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:01:27.716340 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-523971c1-ee23-4d59-a8d1-6e9d939aae50 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:01:27.718305 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-523971c1-ee23-4d59-a8d1-6e9d939aae50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:27.731519 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:01:27.737641 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0206741c-55b3-4812-9361-6e4e729a3adb admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:27.744550 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323666ff-c8f2-415f-ab60-fa80cd9b9038 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b1b651c73664d08accc6edf2f79bace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:27.745338 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323666ff-c8f2-415f-ab60-fa80cd9b9038 service cinder] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:27.745494 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323666ff-c8f2-415f-ab60-fa80cd9b9038 service cinder] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:27.745770 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323666ff-c8f2-415f-ab60-fa80cd9b9038 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:27.747944 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-323666ff-c8f2-415f-ab60-fa80cd9b9038 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:27.826942 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 894/1795] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:01:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:01:27.827309 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523971c1-ee23-4d59-a8d1-6e9d939aae50 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:27.828697 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-523971c1-ee23-4d59-a8d1-6e9d939aae50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:27.862756 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323666ff-c8f2-415f-ab60-fa80cd9b9038 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:27.864437 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323666ff-c8f2-415f-ab60-fa80cd9b9038 service cinder] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:27.876944 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 902/1796] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:01:27 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:01:27.886858 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323666ff-c8f2-415f-ab60-fa80cd9b9038 service cinder] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:27.890541 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-323666ff-c8f2-415f-ab60-fa80cd9b9038 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:27.933579 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523971c1-ee23-4d59-a8d1-6e9d939aae50 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:27.939062 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1340f2-c001-4542-a45b-c52850725f41 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e563fe174db4cc19ae6bf8162cc3d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:27.940017 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1340f2-c001-4542-a45b-c52850725f41 service neutron] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:27.940430 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1340f2-c001-4542-a45b-c52850725f41 service neutron] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:27.940781 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1340f2-c001-4542-a45b-c52850725f41 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:27.954512 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1340f2-c001-4542-a45b-c52850725f41 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:27.956504 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1340f2-c001-4542-a45b-c52850725f41 service neutron] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:27.982748 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 895/1797] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:01:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:01:28.052732 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323666ff-c8f2-415f-ab60-fa80cd9b9038 service cinder] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:28.052732 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c79296a-8f4b-461f-9251-47c1f6158be4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e563fe174db4cc19ae6bf8162cc3d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:28.052732 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c79296a-8f4b-461f-9251-47c1f6158be4 service neutron] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:28.052732 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c79296a-8f4b-461f-9251-47c1f6158be4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:28.053262 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c79296a-8f4b-461f-9251-47c1f6158be4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:28.065326 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c79296a-8f4b-461f-9251-47c1f6158be4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:28.067011 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c79296a-8f4b-461f-9251-47c1f6158be4 service neutron] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:28.078426 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 903/1798] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:01:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:01:28.337755 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1340f2-c001-4542-a45b-c52850725f41 service neutron] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:28.339649 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-cd1340f2-c001-4542-a45b-c52850725f41 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:28.363141 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1340f2-c001-4542-a45b-c52850725f41 service neutron] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:28.372746 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f41379-5e40-402c-9673-b4d5313b2eca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:28.375306 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f41379-5e40-402c-9673-b4d5313b2eca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:28.375855 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f41379-5e40-402c-9673-b4d5313b2eca admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:28.377234 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f41379-5e40-402c-9673-b4d5313b2eca admin admin] PATH_INFO: `/v3/projects/21b141f7b6394853b423e3366c8e8692` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:28.379443 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f41379-5e40-402c-9673-b4d5313b2eca admin admin] RBAC: Authorizing `identity:delete_project(project_id=21b141f7b6394853b423e3366c8e8692)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:28.383741 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f41379-5e40-402c-9673-b4d5313b2eca admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:28.463176 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-f7f41379-5e40-402c-9673-b4d5313b2eca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21b141f7b6394853b423e3366c8e8692', 'request_id': 'req-f7f41379-5e40-402c-9673-b4d5313b2eca'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:28.469915 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c79296a-8f4b-461f-9251-47c1f6158be4 service neutron] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:28.473095 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-3c79296a-8f4b-461f-9251-47c1f6158be4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:28.475095 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-f7f41379-5e40-402c-9673-b4d5313b2eca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21b141f7b6394853b423e3366c8e8692', 'request_id': 'req-f7f41379-5e40-402c-9673-b4d5313b2eca'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:28.485352 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-f7f41379-5e40-402c-9673-b4d5313b2eca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21b141f7b6394853b423e3366c8e8692', 'request_id': 'req-f7f41379-5e40-402c-9673-b4d5313b2eca'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:28.498748 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c79296a-8f4b-461f-9251-47c1f6158be4 service neutron] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:28.520693 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-f7f41379-5e40-402c-9673-b4d5313b2eca admin admin] The token cache is being invalidate because project 21b141f7b6394853b423e3366c8e8692 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 12:01:28.520958 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-f7f41379-5e40-402c-9673-b4d5313b2eca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:28.521658 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-f7f41379-5e40-402c-9673-b4d5313b2eca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:28.523877 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 896/1799] 10.222.0.59 () {58 vars in 1233 bytes} [Tue Oct 11 12:01:28 2022] DELETE /identity/v3/projects/21b141f7b6394853b423e3366c8e8692 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:01:28.528803 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f41379-5e40-402c-9673-b4d5313b2eca admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:28.530316 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-f7f41379-5e40-402c-9673-b4d5313b2eca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:28.587847 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:01:28.598032 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078694c2-a5bb-4567-bcaa-c2f148f4df34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:28.599443 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078694c2-a5bb-4567-bcaa-c2f148f4df34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:28.599957 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078694c2-a5bb-4567-bcaa-c2f148f4df34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:28.600361 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078694c2-a5bb-4567-bcaa-c2f148f4df34 admin admin] PATH_INFO: `/v3/projects/b416a9fa6f7644cab2f7353f91d15ef7` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:28.602351 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078694c2-a5bb-4567-bcaa-c2f148f4df34 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b416a9fa6f7644cab2f7353f91d15ef7)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:28.606281 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078694c2-a5bb-4567-bcaa-c2f148f4df34 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:28.630991 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f41379-5e40-402c-9673-b4d5313b2eca admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:28.638071 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e4f1af-08ee-477d-946f-e5aa674b3a33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:28.638815 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e4f1af-08ee-477d-946f-e5aa674b3a33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:28.639099 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e4f1af-08ee-477d-946f-e5aa674b3a33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:28.639734 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e4f1af-08ee-477d-946f-e5aa674b3a33 admin admin] PATH_INFO: `/v3/users/d7f8df19cc5941688031c3bee68917d0` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:28.640773 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e4f1af-08ee-477d-946f-e5aa674b3a33 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d7f8df19cc5941688031c3bee68917d0)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:28.643174 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e4f1af-08ee-477d-946f-e5aa674b3a33 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:28.660993 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:01:28.690117 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-078694c2-a5bb-4567-bcaa-c2f148f4df34 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b416a9fa6f7644cab2f7353f91d15ef7', 'request_id': 'req-078694c2-a5bb-4567-bcaa-c2f148f4df34'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:28.705358 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-078694c2-a5bb-4567-bcaa-c2f148f4df34 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b416a9fa6f7644cab2f7353f91d15ef7', 'request_id': 'req-078694c2-a5bb-4567-bcaa-c2f148f4df34'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:28.713692 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-b9e4f1af-08ee-477d-946f-e5aa674b3a33 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd7f8df19cc5941688031c3bee68917d0'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:28.721631 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-078694c2-a5bb-4567-bcaa-c2f148f4df34 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b416a9fa6f7644cab2f7353f91d15ef7', 'request_id': 'req-078694c2-a5bb-4567-bcaa-c2f148f4df34'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:28.740467 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-078694c2-a5bb-4567-bcaa-c2f148f4df34 admin admin] The token cache is being invalidate because project b416a9fa6f7644cab2f7353f91d15ef7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81432) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 12:01:28.740788 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-078694c2-a5bb-4567-bcaa-c2f148f4df34 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:28.741500 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-078694c2-a5bb-4567-bcaa-c2f148f4df34 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:28.743952 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 904/1800] 10.222.0.59 () {58 vars in 1233 bytes} [Tue Oct 11 12:01:28 2022] DELETE /identity/v3/projects/b416a9fa6f7644cab2f7353f91d15ef7 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:01:28.746187 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-b9e4f1af-08ee-477d-946f-e5aa674b3a33 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd7f8df19cc5941688031c3bee68917d0'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:28.747284 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078694c2-a5bb-4567-bcaa-c2f148f4df34 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:28.749101 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-078694c2-a5bb-4567-bcaa-c2f148f4df34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:28.760336 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-b9e4f1af-08ee-477d-946f-e5aa674b3a33 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd7f8df19cc5941688031c3bee68917d0'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:28.761351 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-b9e4f1af-08ee-477d-946f-e5aa674b3a33 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd7f8df19cc5941688031c3bee68917d0'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:28.771001 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-b9e4f1af-08ee-477d-946f-e5aa674b3a33 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd7f8df19cc5941688031c3bee68917d0'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:28.779116 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 897/1801] 10.222.0.59 () {58 vars in 1224 bytes} [Tue Oct 11 12:01:28 2022] DELETE /identity/v3/users/d7f8df19cc5941688031c3bee68917d0 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 12:01:28.845241 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078694c2-a5bb-4567-bcaa-c2f148f4df34 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:28.933090 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40d59ad-da1c-4056-accc-ad2cf6b1456d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:28.933753 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40d59ad-da1c-4056-accc-ad2cf6b1456d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:28.934109 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40d59ad-da1c-4056-accc-ad2cf6b1456d admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:28.934700 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40d59ad-da1c-4056-accc-ad2cf6b1456d admin admin] PATH_INFO: `/v3/users/f5fa55a8a7a247959550948f115f866a` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:28.935999 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40d59ad-da1c-4056-accc-ad2cf6b1456d admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5fa55a8a7a247959550948f115f866a)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:28.937986 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40d59ad-da1c-4056-accc-ad2cf6b1456d admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:29.044842 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-a40d59ad-da1c-4056-accc-ad2cf6b1456d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5fa55a8a7a247959550948f115f866a'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:29.093044 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:DELETE=15,keystone:SELECT=84,keystone:INSERT=1 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:01:29.095803 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-a40d59ad-da1c-4056-accc-ad2cf6b1456d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5fa55a8a7a247959550948f115f866a'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:29.096715 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-a40d59ad-da1c-4056-accc-ad2cf6b1456d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5fa55a8a7a247959550948f115f866a'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:29.115966 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-a40d59ad-da1c-4056-accc-ad2cf6b1456d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5fa55a8a7a247959550948f115f866a'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:29.117702 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-a40d59ad-da1c-4056-accc-ad2cf6b1456d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5fa55a8a7a247959550948f115f866a'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:29.143116 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 905/1802] 10.222.0.59 () {58 vars in 1224 bytes} [Tue Oct 11 12:01:28 2022] DELETE /identity/v3/users/f5fa55a8a7a247959550948f115f866a => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:01:29.251033 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e4f1af-08ee-477d-946f-e5aa674b3a33 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:29.253917 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-b9e4f1af-08ee-477d-946f-e5aa674b3a33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:29.306111 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e4f1af-08ee-477d-946f-e5aa674b3a33 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:29.364589 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40d59ad-da1c-4056-accc-ad2cf6b1456d admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:29.366965 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-a40d59ad-da1c-4056-accc-ad2cf6b1456d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:29.468423 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1f70bb-3e23-4991-bc17-7052277ac4a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:29.469975 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1f70bb-3e23-4991-bc17-7052277ac4a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:29.470456 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1f70bb-3e23-4991-bc17-7052277ac4a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:29.471139 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1f70bb-3e23-4991-bc17-7052277ac4a5 admin admin] PATH_INFO: `/v3/projects/af431dae2476490196b40d063e704a84` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:29.473521 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1f70bb-3e23-4991-bc17-7052277ac4a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=af431dae2476490196b40d063e704a84)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:29.479424 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1f70bb-3e23-4991-bc17-7052277ac4a5 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:29.545592 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40d59ad-da1c-4056-accc-ad2cf6b1456d admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:29.562820 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1bb00e-2a09-42cb-9d53-7f665c79eaa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:29.563544 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1bb00e-2a09-42cb-9d53-7f665c79eaa7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:29.563836 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1bb00e-2a09-42cb-9d53-7f665c79eaa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:29.564116 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1bb00e-2a09-42cb-9d53-7f665c79eaa7 admin admin] PATH_INFO: `/v3/users/f8a601880c6640a0ac0dd6b6299ac4a9` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:29.572120 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1bb00e-2a09-42cb-9d53-7f665c79eaa7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8a601880c6640a0ac0dd6b6299ac4a9)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:29.576294 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1bb00e-2a09-42cb-9d53-7f665c79eaa7 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:29.632155 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-cc1f70bb-3e23-4991-bc17-7052277ac4a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af431dae2476490196b40d063e704a84', 'request_id': 'req-cc1f70bb-3e23-4991-bc17-7052277ac4a5'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:29.649942 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-cc1f70bb-3e23-4991-bc17-7052277ac4a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af431dae2476490196b40d063e704a84', 'request_id': 'req-cc1f70bb-3e23-4991-bc17-7052277ac4a5'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:29.662012 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:01:29.674094 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-cc1f70bb-3e23-4991-bc17-7052277ac4a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af431dae2476490196b40d063e704a84', 'request_id': 'req-cc1f70bb-3e23-4991-bc17-7052277ac4a5'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:29.684504 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-5c1bb00e-2a09-42cb-9d53-7f665c79eaa7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8a601880c6640a0ac0dd6b6299ac4a9'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:29.702446 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-cc1f70bb-3e23-4991-bc17-7052277ac4a5 admin admin] The token cache is being invalidate because project af431dae2476490196b40d063e704a84 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 12:01:29.702806 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-cc1f70bb-3e23-4991-bc17-7052277ac4a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:29.703976 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-cc1f70bb-3e23-4991-bc17-7052277ac4a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:29.713592 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 898/1803] 10.222.0.59 () {58 vars in 1233 bytes} [Tue Oct 11 12:01:29 2022] DELETE /identity/v3/projects/af431dae2476490196b40d063e704a84 => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:01:29.715235 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1f70bb-3e23-4991-bc17-7052277ac4a5 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:29.720932 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-cc1f70bb-3e23-4991-bc17-7052277ac4a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:29.722583 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-5c1bb00e-2a09-42cb-9d53-7f665c79eaa7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8a601880c6640a0ac0dd6b6299ac4a9'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:29.724039 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-5c1bb00e-2a09-42cb-9d53-7f665c79eaa7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8a601880c6640a0ac0dd6b6299ac4a9'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:29.754916 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-5c1bb00e-2a09-42cb-9d53-7f665c79eaa7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8a601880c6640a0ac0dd6b6299ac4a9'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:29.757069 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-5c1bb00e-2a09-42cb-9d53-7f665c79eaa7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8a601880c6640a0ac0dd6b6299ac4a9'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:29.774474 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 906/1804] 10.222.0.59 () {58 vars in 1224 bytes} [Tue Oct 11 12:01:29 2022] DELETE /identity/v3/users/f8a601880c6640a0ac0dd6b6299ac4a9 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:01:29.776098 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dffce7-1020-40cc-84b5-cc58f082d3ce None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:29.776488 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dffce7-1020-40cc-84b5-cc58f082d3ce None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:29.777251 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dffce7-1020-40cc-84b5-cc58f082d3ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:29.809711 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-85dffce7-1020-40cc-84b5-cc58f082d3ce None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:01:29.811716 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-85dffce7-1020-40cc-84b5-cc58f082d3ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:29.891424 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:01:29.892278 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1f70bb-3e23-4991-bc17-7052277ac4a5 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:29.904378 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 907/1805] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:01:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:01:29.906598 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573752f0-96e1-4db3-86ec-9a6f222843a6 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:29.907462 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573752f0-96e1-4db3-86ec-9a6f222843a6 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:29.908031 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573752f0-96e1-4db3-86ec-9a6f222843a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:29.945123 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-573752f0-96e1-4db3-86ec-9a6f222843a6 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:01:29.949369 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-573752f0-96e1-4db3-86ec-9a6f222843a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:29.980912 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 908/1806] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:01:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:01:29.980912 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a20d35f-4f52-4603-83cf-c49326026e57 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:29.980912 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a20d35f-4f52-4603-83cf-c49326026e57 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:29.980912 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a20d35f-4f52-4603-83cf-c49326026e57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:29.995219 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f3ff2f-8cac-45ca-8508-2c6dc91d380f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:29.996006 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f3ff2f-8cac-45ca-8508-2c6dc91d380f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:29.997355 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f3ff2f-8cac-45ca-8508-2c6dc91d380f admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:29.997655 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f3ff2f-8cac-45ca-8508-2c6dc91d380f admin admin] PATH_INFO: `/v3/projects/9a54b2b3fce8405080248bbb889c67bf` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:29.999010 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f3ff2f-8cac-45ca-8508-2c6dc91d380f admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a54b2b3fce8405080248bbb889c67bf)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:30.001127 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f3ff2f-8cac-45ca-8508-2c6dc91d380f admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:30.017168 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-1a20d35f-4f52-4603-83cf-c49326026e57 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:01:30.019668 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-1a20d35f-4f52-4603-83cf-c49326026e57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:30.040318 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 909/1807] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:01:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:01:30.041503 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a20d35f-4f52-4603-83cf-c49326026e57 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:30.043765 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-1a20d35f-4f52-4603-83cf-c49326026e57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:30.076115 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-90f3ff2f-8cac-45ca-8508-2c6dc91d380f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a54b2b3fce8405080248bbb889c67bf', 'request_id': 'req-90f3ff2f-8cac-45ca-8508-2c6dc91d380f'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:30.083357 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-90f3ff2f-8cac-45ca-8508-2c6dc91d380f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a54b2b3fce8405080248bbb889c67bf', 'request_id': 'req-90f3ff2f-8cac-45ca-8508-2c6dc91d380f'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:30.091974 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-90f3ff2f-8cac-45ca-8508-2c6dc91d380f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a54b2b3fce8405080248bbb889c67bf', 'request_id': 'req-90f3ff2f-8cac-45ca-8508-2c6dc91d380f'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:30.107215 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-90f3ff2f-8cac-45ca-8508-2c6dc91d380f admin admin] The token cache is being invalidate because project 9a54b2b3fce8405080248bbb889c67bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 12:01:30.107513 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-90f3ff2f-8cac-45ca-8508-2c6dc91d380f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:30.108232 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-90f3ff2f-8cac-45ca-8508-2c6dc91d380f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:30.111490 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 899/1808] 10.222.0.59 () {58 vars in 1233 bytes} [Tue Oct 11 12:01:29 2022] DELETE /identity/v3/projects/9a54b2b3fce8405080248bbb889c67bf => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:01:30.114432 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f3ff2f-8cac-45ca-8508-2c6dc91d380f admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:30.116255 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-90f3ff2f-8cac-45ca-8508-2c6dc91d380f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:30.148241 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a20d35f-4f52-4603-83cf-c49326026e57 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:30.161278 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:01:30.211808 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f3ff2f-8cac-45ca-8508-2c6dc91d380f admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:30.217084 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2be7e8a-eae3-4aac-91f0-881f16fb4c1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:30.217843 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2be7e8a-eae3-4aac-91f0-881f16fb4c1c admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:30.218235 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2be7e8a-eae3-4aac-91f0-881f16fb4c1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:30.218659 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2be7e8a-eae3-4aac-91f0-881f16fb4c1c admin admin] PATH_INFO: `/v3/domains` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:30.219368 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2be7e8a-eae3-4aac-91f0-881f16fb4c1c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 12:01:30.219698 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2be7e8a-eae3-4aac-91f0-881f16fb4c1c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:30.221473 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2be7e8a-eae3-4aac-91f0-881f16fb4c1c admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:30.229260 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c93c3e-4c7a-4fe2-849d-21984da832ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e563fe174db4cc19ae6bf8162cc3d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:30.229885 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c93c3e-4c7a-4fe2-849d-21984da832ae service neutron] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:30.230712 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c93c3e-4c7a-4fe2-849d-21984da832ae service neutron] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:30.230712 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c93c3e-4c7a-4fe2-849d-21984da832ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:30.232350 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-b3c93c3e-4c7a-4fe2-849d-21984da832ae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:30.236336 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 900/1809] 10.222.0.59 () {58 vars in 1153 bytes} [Tue Oct 11 12:01:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 12:01:30.236336 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14a8b65-fc0f-46e5-bce4-176e20234bf8 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:30.236336 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14a8b65-fc0f-46e5-bce4-176e20234bf8 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:30.236336 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14a8b65-fc0f-46e5-bce4-176e20234bf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:30.245981 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c93c3e-4c7a-4fe2-849d-21984da832ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:30.247504 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c93c3e-4c7a-4fe2-849d-21984da832ae service neutron] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:30.258136 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 910/1810] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:01:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:01:30.259091 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c93c3e-4c7a-4fe2-849d-21984da832ae service neutron] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:30.260534 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-c14a8b65-fc0f-46e5-bce4-176e20234bf8 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:01:30.261804 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-c14a8b65-fc0f-46e5-bce4-176e20234bf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:30.269156 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c93c3e-4c7a-4fe2-849d-21984da832ae service neutron] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:30.272266 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 901/1811] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:01:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:01:30.274714 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003b70a4-eced-4fb0-b492-6ea18c9570ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:30.275354 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003b70a4-eced-4fb0-b492-6ea18c9570ab admin admin] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:30.275654 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003b70a4-eced-4fb0-b492-6ea18c9570ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:30.275985 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003b70a4-eced-4fb0-b492-6ea18c9570ab admin admin] PATH_INFO: `/v3/projects` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:30.277108 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003b70a4-eced-4fb0-b492-6ea18c9570ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:30.279142 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003b70a4-eced-4fb0-b492-6ea18c9570ab admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:30.282212 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf99c02-eb6b-43f7-a9f8-526a22064533 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:30.282639 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf99c02-eb6b-43f7-a9f8-526a22064533 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:30.282934 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf99c02-eb6b-43f7-a9f8-526a22064533 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:30.314330 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 911/1812] 10.222.0.59 () {60 vars in 1153 bytes} [Tue Oct 11 12:01:30 2022] POST /identity/v3/projects => generated 385 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:01:30.315615 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003b70a4-eced-4fb0-b492-6ea18c9570ab admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:30.315933 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-bcf99c02-eb6b-43f7-a9f8-526a22064533 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:01:30.317037 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-bcf99c02-eb6b-43f7-a9f8-526a22064533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:30.421708 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 902/1813] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:01:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 12:01:30.425559 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003b70a4-eced-4fb0-b492-6ea18c9570ab admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:30.426649 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf99c02-eb6b-43f7-a9f8-526a22064533 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:30.432532 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6602b95-f045-4b70-a959-2add11c6f608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:30.433172 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6602b95-f045-4b70-a959-2add11c6f608 admin admin] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:30.433615 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6602b95-f045-4b70-a959-2add11c6f608 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:30.433895 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6602b95-f045-4b70-a959-2add11c6f608 admin admin] PATH_INFO: `/v3/users` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:30.434646 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6602b95-f045-4b70-a959-2add11c6f608 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:30.441560 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf99c02-eb6b-43f7-a9f8-526a22064533 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:30.444668 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6602b95-f045-4b70-a959-2add11c6f608 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:30.452243 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a37be2-37fb-43f0-aacf-1d73291014bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:30.453305 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a37be2-37fb-43f0-aacf-1d73291014bf admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:30.453786 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a37be2-37fb-43f0-aacf-1d73291014bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:30.454447 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a37be2-37fb-43f0-aacf-1d73291014bf admin admin] PATH_INFO: `/v3/domains` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:30.455345 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a37be2-37fb-43f0-aacf-1d73291014bf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 12:01:30.455766 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a37be2-37fb-43f0-aacf-1d73291014bf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:30.457416 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a37be2-37fb-43f0-aacf-1d73291014bf admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:30.477653 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 903/1814] 10.222.0.59 () {58 vars in 1153 bytes} [Tue Oct 11 12:01:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 12:01:30.480247 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 912/1815] 10.222.0.59 () {60 vars in 1144 bytes} [Tue Oct 11 12:01:30 2022] POST /identity/v3/users => generated 293 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:01:30.482270 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a37be2-37fb-43f0-aacf-1d73291014bf admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:30.484932 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6602b95-f045-4b70-a959-2add11c6f608 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:30.498625 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6602b95-f045-4b70-a959-2add11c6f608 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:30.499107 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a37be2-37fb-43f0-aacf-1d73291014bf admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:30.505151 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61507fad-fd45-4a03-a21b-2144d1412ece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:30.505948 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61507fad-fd45-4a03-a21b-2144d1412ece admin admin] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:30.506572 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61507fad-fd45-4a03-a21b-2144d1412ece admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:30.506956 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61507fad-fd45-4a03-a21b-2144d1412ece admin admin] PATH_INFO: `/v3/projects` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:30.507414 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c60caf1-ca37-4b08-b9f9-5eab6fd838db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:30.507828 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61507fad-fd45-4a03-a21b-2144d1412ece admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:30.508482 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c60caf1-ca37-4b08-b9f9-5eab6fd838db admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:30.508973 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c60caf1-ca37-4b08-b9f9-5eab6fd838db admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:30.509309 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c60caf1-ca37-4b08-b9f9-5eab6fd838db admin admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:30.509905 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61507fad-fd45-4a03-a21b-2144d1412ece admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:30.510868 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c60caf1-ca37-4b08-b9f9-5eab6fd838db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:30.515023 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c60caf1-ca37-4b08-b9f9-5eab6fd838db admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:30.540313 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 913/1816] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:01:30 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:01:30.544262 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c60caf1-ca37-4b08-b9f9-5eab6fd838db admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:30.557234 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:01:30.558066 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 904/1817] 10.222.0.59 () {60 vars in 1153 bytes} [Tue Oct 11 12:01:30 2022] POST /identity/v3/projects => generated 375 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:01:30.566979 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61507fad-fd45-4a03-a21b-2144d1412ece admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:30.567248 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c60caf1-ca37-4b08-b9f9-5eab6fd838db admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:30.643910 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce253c4-ac9d-4ffa-9e10-c5366843bb52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:30.644676 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce253c4-ac9d-4ffa-9e10-c5366843bb52 admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:30.645029 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce253c4-ac9d-4ffa-9e10-c5366843bb52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:30.645375 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce253c4-ac9d-4ffa-9e10-c5366843bb52 admin admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:30.646123 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce253c4-ac9d-4ffa-9e10-c5366843bb52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:30.648067 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce253c4-ac9d-4ffa-9e10-c5366843bb52 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:30.650478 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61507fad-fd45-4a03-a21b-2144d1412ece admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:30.659417 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bac493e-d2b6-4ff7-9aa5-137cabd90980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:30.660073 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bac493e-d2b6-4ff7-9aa5-137cabd90980 admin admin] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:30.660365 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bac493e-d2b6-4ff7-9aa5-137cabd90980 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:30.660728 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bac493e-d2b6-4ff7-9aa5-137cabd90980 admin admin] PATH_INFO: `/v3/users` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:30.661569 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bac493e-d2b6-4ff7-9aa5-137cabd90980 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:30.663749 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bac493e-d2b6-4ff7-9aa5-137cabd90980 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:30.664680 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 914/1818] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:01:30 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:01:30.670615 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce253c4-ac9d-4ffa-9e10-c5366843bb52 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:30.694231 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce253c4-ac9d-4ffa-9e10-c5366843bb52 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:30.697970 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 905/1819] 10.222.0.59 () {60 vars in 1144 bytes} [Tue Oct 11 12:01:30 2022] POST /identity/v3/users => generated 288 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:01:30.705187 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bac493e-d2b6-4ff7-9aa5-137cabd90980 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:30.705462 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac4bdde-b97f-4ab0-8d9d-ed5e6be20533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:30.706266 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac4bdde-b97f-4ab0-8d9d-ed5e6be20533 admin admin] REQUEST_METHOD: `PUT` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:30.706822 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac4bdde-b97f-4ab0-8d9d-ed5e6be20533 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:30.707262 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac4bdde-b97f-4ab0-8d9d-ed5e6be20533 admin admin] PATH_INFO: `/v3/projects/48276d0b291e47038b0cbabd5b68952a/users/de67d44b887c4a868c8bb13343e5f86c/roles/de2d2bd26fff4748a79990f97a107164` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:30.730985 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bac493e-d2b6-4ff7-9aa5-137cabd90980 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:30.735861 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac4bdde-b97f-4ab0-8d9d-ed5e6be20533 admin admin] RBAC: Authorizing `identity:create_grant(project_id=48276d0b291e47038b0cbabd5b68952a, user_id=de67d44b887c4a868c8bb13343e5f86c, role_id=de2d2bd26fff4748a79990f97a107164)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:30.739909 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df06cf2-4920-47de-9f27-fe5a5ac4d4fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:30.740694 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df06cf2-4920-47de-9f27-fe5a5ac4d4fe admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:30.741077 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df06cf2-4920-47de-9f27-fe5a5ac4d4fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:30.741547 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df06cf2-4920-47de-9f27-fe5a5ac4d4fe admin admin] PATH_INFO: `/v3/roles` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:30.743344 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df06cf2-4920-47de-9f27-fe5a5ac4d4fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:30.743575 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac4bdde-b97f-4ab0-8d9d-ed5e6be20533 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:30.748153 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df06cf2-4920-47de-9f27-fe5a5ac4d4fe admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:30.754458 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:01:30.779302 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 906/1821] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:01:30 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:01:30.783413 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 915/1821] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 12:01:30 2022] PUT /identity/v3/projects/48276d0b291e47038b0cbabd5b68952a/users/de67d44b887c4a868c8bb13343e5f86c/roles/de2d2bd26fff4748a79990f97a107164 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:01:30.796922 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac4bdde-b97f-4ab0-8d9d-ed5e6be20533 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:30.797804 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d8f033-b298-4487-b59d-65de15669e1e None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:30.798292 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d8f033-b298-4487-b59d-65de15669e1e None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:30.798821 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d8f033-b298-4487-b59d-65de15669e1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:30.856835 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-a7d8f033-b298-4487-b59d-65de15669e1e None None] MFA Rules not processed for user `de67d44b887c4a868c8bb13343e5f86c`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:01:30.858184 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-a7d8f033-b298-4487-b59d-65de15669e1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:30.965329 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac4bdde-b97f-4ab0-8d9d-ed5e6be20533 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:30.977926 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741b0609-da08-461f-8e7b-84a972857c0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:30.986778 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741b0609-da08-461f-8e7b-84a972857c0e admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:30.986778 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741b0609-da08-461f-8e7b-84a972857c0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:30.986778 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741b0609-da08-461f-8e7b-84a972857c0e admin admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:30.991105 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741b0609-da08-461f-8e7b-84a972857c0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:30.993053 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741b0609-da08-461f-8e7b-84a972857c0e admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:31.012185 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 916/1822] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:01:30 2022] GET /identity/v3/roles => generated 1420 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:01:31.015458 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741b0609-da08-461f-8e7b-84a972857c0e admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:31.017288 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-741b0609-da08-461f-8e7b-84a972857c0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:31.036870 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741b0609-da08-461f-8e7b-84a972857c0e admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:31.041341 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 907/1823] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:01:30 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:01:31.042639 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d8f033-b298-4487-b59d-65de15669e1e None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:31.046825 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689d7408-d1e8-4063-a154-c37d56add70b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:31.047755 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689d7408-d1e8-4063-a154-c37d56add70b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:31.048118 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689d7408-d1e8-4063-a154-c37d56add70b admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:31.048399 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689d7408-d1e8-4063-a154-c37d56add70b admin admin] PATH_INFO: `/v3/projects/0f914c3c6fa84cfaa587ed1a9e9ef1e2` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:31.049874 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689d7408-d1e8-4063-a154-c37d56add70b admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f914c3c6fa84cfaa587ed1a9e9ef1e2)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:31.052730 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689d7408-d1e8-4063-a154-c37d56add70b admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:31.058472 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d8f033-b298-4487-b59d-65de15669e1e None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:31.065400 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308802ee-0428-455d-958a-1a18181724f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:31.067131 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308802ee-0428-455d-958a-1a18181724f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:31.067432 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308802ee-0428-455d-958a-1a18181724f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:31.068298 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308802ee-0428-455d-958a-1a18181724f9 admin admin] PATH_INFO: `/v3/projects/6d67d978bce04debbd1ce49e589a735a/users/1b664bef72e64f508d2cc56fd6a210fd/roles/de2d2bd26fff4748a79990f97a107164` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:31.110898 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308802ee-0428-455d-958a-1a18181724f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d67d978bce04debbd1ce49e589a735a, user_id=1b664bef72e64f508d2cc56fd6a210fd, role_id=de2d2bd26fff4748a79990f97a107164)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:31.117830 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308802ee-0428-455d-958a-1a18181724f9 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:31.127007 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-689d7408-d1e8-4063-a154-c37d56add70b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f914c3c6fa84cfaa587ed1a9e9ef1e2', 'request_id': 'req-689d7408-d1e8-4063-a154-c37d56add70b'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:31.137466 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-689d7408-d1e8-4063-a154-c37d56add70b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f914c3c6fa84cfaa587ed1a9e9ef1e2', 'request_id': 'req-689d7408-d1e8-4063-a154-c37d56add70b'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:31.152865 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 908/1824] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 12:01:31 2022] PUT /identity/v3/projects/6d67d978bce04debbd1ce49e589a735a/users/1b664bef72e64f508d2cc56fd6a210fd/roles/de2d2bd26fff4748a79990f97a107164 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:01:31.154829 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308802ee-0428-455d-958a-1a18181724f9 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:31.166874 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-689d7408-d1e8-4063-a154-c37d56add70b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f914c3c6fa84cfaa587ed1a9e9ef1e2', 'request_id': 'req-689d7408-d1e8-4063-a154-c37d56add70b'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:31.186297 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:01:31.212879 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-689d7408-d1e8-4063-a154-c37d56add70b admin admin] The token cache is being invalidate because project 0f914c3c6fa84cfaa587ed1a9e9ef1e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81432) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 12:01:31.213180 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-689d7408-d1e8-4063-a154-c37d56add70b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:31.215260 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-689d7408-d1e8-4063-a154-c37d56add70b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:31.222682 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 917/1825] 10.222.0.59 () {58 vars in 1233 bytes} [Tue Oct 11 12:01:31 2022] DELETE /identity/v3/projects/0f914c3c6fa84cfaa587ed1a9e9ef1e2 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:01:31.222682 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a0d23c-c19e-45c6-ab60-81701e7e3d6e None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:31.222682 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a0d23c-c19e-45c6-ab60-81701e7e3d6e None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:31.222682 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a0d23c-c19e-45c6-ab60-81701e7e3d6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:31.312300 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308802ee-0428-455d-958a-1a18181724f9 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:31.320988 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-67a0d23c-c19e-45c6-ab60-81701e7e3d6e None None] MFA Rules not processed for user `1b664bef72e64f508d2cc56fd6a210fd`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:01:31.327265 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60eba7e5-aed7-4e15-8008-8eebed602408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:31.328364 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-67a0d23c-c19e-45c6-ab60-81701e7e3d6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:31.328777 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eba7e5-aed7-4e15-8008-8eebed602408 admin admin] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:31.329437 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eba7e5-aed7-4e15-8008-8eebed602408 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:31.329976 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eba7e5-aed7-4e15-8008-8eebed602408 admin admin] PATH_INFO: `/v3/projects` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:31.331254 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60eba7e5-aed7-4e15-8008-8eebed602408 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:31.337016 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60eba7e5-aed7-4e15-8008-8eebed602408 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:31.417160 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 909/1826] 10.222.0.59 () {60 vars in 1153 bytes} [Tue Oct 11 12:01:31 2022] POST /identity/v3/projects => generated 385 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:01:31.419619 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60eba7e5-aed7-4e15-8008-8eebed602408 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:31.421947 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-60eba7e5-aed7-4e15-8008-8eebed602408 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:31.488942 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 918/1827] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:01:31 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:01:31.490778 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a0d23c-c19e-45c6-ab60-81701e7e3d6e None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:31.496125 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-67a0d23c-c19e-45c6-ab60-81701e7e3d6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:31.521622 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:01:31.534472 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a0d23c-c19e-45c6-ab60-81701e7e3d6e None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:31.539754 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d51498-f575-4df2-b519-f400ee084aae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:31.540383 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d51498-f575-4df2-b519-f400ee084aae admin admin] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:31.540847 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d51498-f575-4df2-b519-f400ee084aae admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:31.541276 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d51498-f575-4df2-b519-f400ee084aae admin admin] PATH_INFO: `/v3/users` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:31.542371 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d51498-f575-4df2-b519-f400ee084aae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:31.544802 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d51498-f575-4df2-b519-f400ee084aae admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:31.555267 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60eba7e5-aed7-4e15-8008-8eebed602408 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:31.574218 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d24bf3-52a3-4706-ab31-2471b32db5cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:31.575221 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d24bf3-52a3-4706-ab31-2471b32db5cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:31.575731 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d24bf3-52a3-4706-ab31-2471b32db5cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:31.576284 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d24bf3-52a3-4706-ab31-2471b32db5cf admin admin] PATH_INFO: `/v3/users/17dbccf6199140a3839fc6e176643648` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:31.579290 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d24bf3-52a3-4706-ab31-2471b32db5cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=17dbccf6199140a3839fc6e176643648)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:31.584262 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 919/1828] 10.222.0.59 () {60 vars in 1144 bytes} [Tue Oct 11 12:01:31 2022] POST /identity/v3/users => generated 299 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:01:31.585019 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d24bf3-52a3-4706-ab31-2471b32db5cf admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:31.587197 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d51498-f575-4df2-b519-f400ee084aae admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:31.608132 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d51498-f575-4df2-b519-f400ee084aae admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:31.619930 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890e98dd-7399-4157-8c20-86a2c8a30bab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:31.620849 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890e98dd-7399-4157-8c20-86a2c8a30bab admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:31.621406 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890e98dd-7399-4157-8c20-86a2c8a30bab admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:31.621930 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890e98dd-7399-4157-8c20-86a2c8a30bab admin admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:31.622990 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890e98dd-7399-4157-8c20-86a2c8a30bab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:31.625345 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890e98dd-7399-4157-8c20-86a2c8a30bab admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:31.653809 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 920/1829] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:01:31 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:01:31.655577 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890e98dd-7399-4157-8c20-86a2c8a30bab admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:31.658357 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-890e98dd-7399-4157-8c20-86a2c8a30bab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:31.670978 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-06d24bf3-52a3-4706-ab31-2471b32db5cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '17dbccf6199140a3839fc6e176643648'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:31.703451 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-06d24bf3-52a3-4706-ab31-2471b32db5cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '17dbccf6199140a3839fc6e176643648'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:31.714727 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-06d24bf3-52a3-4706-ab31-2471b32db5cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '17dbccf6199140a3839fc6e176643648'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:31.715900 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-06d24bf3-52a3-4706-ab31-2471b32db5cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '17dbccf6199140a3839fc6e176643648'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:31.728761 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-06d24bf3-52a3-4706-ab31-2471b32db5cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '17dbccf6199140a3839fc6e176643648'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:31.737694 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 910/1830] 10.222.0.59 () {58 vars in 1224 bytes} [Tue Oct 11 12:01:31 2022] DELETE /identity/v3/users/17dbccf6199140a3839fc6e176643648 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:01:31.738937 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d24bf3-52a3-4706-ab31-2471b32db5cf admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:31.742276 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-06d24bf3-52a3-4706-ab31-2471b32db5cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:31.811572 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890e98dd-7399-4157-8c20-86a2c8a30bab admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:31.835572 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d24bf3-52a3-4706-ab31-2471b32db5cf admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:31.841468 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d79275c-12e1-4e77-8aad-30e5012c6dcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:31.842373 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d79275c-12e1-4e77-8aad-30e5012c6dcd admin admin] REQUEST_METHOD: `PUT` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:31.843305 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d79275c-12e1-4e77-8aad-30e5012c6dcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:31.843901 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d79275c-12e1-4e77-8aad-30e5012c6dcd admin admin] PATH_INFO: `/v3/projects/583873d675d24d0b942029f1614dd31d/users/5cfb1872b5384903bbe48686a1a58879/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:31.867289 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d79275c-12e1-4e77-8aad-30e5012c6dcd admin admin] RBAC: Authorizing `identity:create_grant(project_id=583873d675d24d0b942029f1614dd31d, user_id=5cfb1872b5384903bbe48686a1a58879, role_id=5cf1f4bd2fef46ab9c6eff56d52f9f69)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:31.870599 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d79275c-12e1-4e77-8aad-30e5012c6dcd admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:31.898829 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 911/1831] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 12:01:31 2022] PUT /identity/v3/projects/583873d675d24d0b942029f1614dd31d/users/5cfb1872b5384903bbe48686a1a58879/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:01:31.904893 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d79275c-12e1-4e77-8aad-30e5012c6dcd admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:31.922152 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853114c3-0fce-4c5f-8846-6a19ec776310 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b1b651c73664d08accc6edf2f79bace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:31.923114 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853114c3-0fce-4c5f-8846-6a19ec776310 service cinder] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:31.923731 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853114c3-0fce-4c5f-8846-6a19ec776310 service cinder] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:31.924109 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853114c3-0fce-4c5f-8846-6a19ec776310 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:31.926507 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-853114c3-0fce-4c5f-8846-6a19ec776310 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:31.977067 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:01:32.028171 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853114c3-0fce-4c5f-8846-6a19ec776310 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:32.029666 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853114c3-0fce-4c5f-8846-6a19ec776310 service cinder] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:32.033496 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:01:32.041714 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 921/1832] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:01:31 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:01:32.047727 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d79275c-12e1-4e77-8aad-30e5012c6dcd admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:32.056232 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d646ffb9-7bc6-4243-9fc4-6a2ce58c18ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:32.056864 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d646ffb9-7bc6-4243-9fc4-6a2ce58c18ed admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:32.057143 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d646ffb9-7bc6-4243-9fc4-6a2ce58c18ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:32.057478 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d646ffb9-7bc6-4243-9fc4-6a2ce58c18ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:32.058124 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d646ffb9-7bc6-4243-9fc4-6a2ce58c18ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:32.059823 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d646ffb9-7bc6-4243-9fc4-6a2ce58c18ed admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:32.073358 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 912/1833] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:01:31 2022] GET /identity/v3/roles => generated 1420 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 11 12:01:32.077886 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853114c3-0fce-4c5f-8846-6a19ec776310 service cinder] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:32.090695 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853114c3-0fce-4c5f-8846-6a19ec776310 service cinder] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:32.096721 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab79449-35c1-43fd-bccc-6af3eec86935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:32.097571 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab79449-35c1-43fd-bccc-6af3eec86935 admin admin] REQUEST_METHOD: `PUT` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:32.098307 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab79449-35c1-43fd-bccc-6af3eec86935 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:32.098307 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab79449-35c1-43fd-bccc-6af3eec86935 admin admin] PATH_INFO: `/v3/projects/583873d675d24d0b942029f1614dd31d/users/5cfb1872b5384903bbe48686a1a58879/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:32.100577 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab79449-35c1-43fd-bccc-6af3eec86935 admin admin] RBAC: Authorizing `identity:create_grant(project_id=583873d675d24d0b942029f1614dd31d, user_id=5cfb1872b5384903bbe48686a1a58879, role_id=5cf1f4bd2fef46ab9c6eff56d52f9f69)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:32.103723 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab79449-35c1-43fd-bccc-6af3eec86935 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:32.120361 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 922/1834] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 12:01:32 2022] PUT /identity/v3/projects/583873d675d24d0b942029f1614dd31d/users/5cfb1872b5384903bbe48686a1a58879/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 12:01:32.131955 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37147fe-12f5-40c1-b436-329a8a8fb7fd None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:32.132358 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37147fe-12f5-40c1-b436-329a8a8fb7fd None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:32.133080 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37147fe-12f5-40c1-b436-329a8a8fb7fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:32.171255 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-b37147fe-12f5-40c1-b436-329a8a8fb7fd None None] MFA Rules not processed for user `5cfb1872b5384903bbe48686a1a58879`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:01:32.174118 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-b37147fe-12f5-40c1-b436-329a8a8fb7fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:32.237919 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab79449-35c1-43fd-bccc-6af3eec86935 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:32.240835 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-cab79449-35c1-43fd-bccc-6af3eec86935 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:32.299642 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 913/1835] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:01:32 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:01:32.318427 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37147fe-12f5-40c1-b436-329a8a8fb7fd None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:32.320207 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-b37147fe-12f5-40c1-b436-329a8a8fb7fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:32.376617 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab79449-35c1-43fd-bccc-6af3eec86935 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:32.386432 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cc869b-c8e3-48bf-b693-2fcc3b37e4b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:32.387210 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cc869b-c8e3-48bf-b693-2fcc3b37e4b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:32.387487 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cc869b-c8e3-48bf-b693-2fcc3b37e4b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:32.387756 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cc869b-c8e3-48bf-b693-2fcc3b37e4b3 admin admin] PATH_INFO: `/v3/projects/45f5dd0d6f4044689b2babcb323d2b5b` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:32.389942 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cc869b-c8e3-48bf-b693-2fcc3b37e4b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=45f5dd0d6f4044689b2babcb323d2b5b)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:32.393006 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cc869b-c8e3-48bf-b693-2fcc3b37e4b3 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:32.409339 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37147fe-12f5-40c1-b436-329a8a8fb7fd None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:32.413554 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3bd43f-f84b-430b-b055-583ab4a84719 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b1b651c73664d08accc6edf2f79bace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:32.414329 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3bd43f-f84b-430b-b055-583ab4a84719 service cinder] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:32.414704 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3bd43f-f84b-430b-b055-583ab4a84719 service cinder] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:32.414978 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3bd43f-f84b-430b-b055-583ab4a84719 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:32.428663 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3bd43f-f84b-430b-b055-583ab4a84719 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:32.430078 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3bd43f-f84b-430b-b055-583ab4a84719 service cinder] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:32.441797 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 914/1836] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:01:32 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 11 12:01:32.458712 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-b6cc869b-c8e3-48bf-b693-2fcc3b37e4b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45f5dd0d6f4044689b2babcb323d2b5b', 'request_id': 'req-b6cc869b-c8e3-48bf-b693-2fcc3b37e4b3'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:32.470971 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-b6cc869b-c8e3-48bf-b693-2fcc3b37e4b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45f5dd0d6f4044689b2babcb323d2b5b', 'request_id': 'req-b6cc869b-c8e3-48bf-b693-2fcc3b37e4b3'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:32.481829 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-b6cc869b-c8e3-48bf-b693-2fcc3b37e4b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45f5dd0d6f4044689b2babcb323d2b5b', 'request_id': 'req-b6cc869b-c8e3-48bf-b693-2fcc3b37e4b3'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:32.511343 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-b6cc869b-c8e3-48bf-b693-2fcc3b37e4b3 admin admin] The token cache is being invalidate because project 45f5dd0d6f4044689b2babcb323d2b5b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81432) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 12:01:32.511883 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-b6cc869b-c8e3-48bf-b693-2fcc3b37e4b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:32.513327 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-b6cc869b-c8e3-48bf-b693-2fcc3b37e4b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:32.516868 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 923/1837] 10.222.0.59 () {58 vars in 1233 bytes} [Tue Oct 11 12:01:32 2022] DELETE /identity/v3/projects/45f5dd0d6f4044689b2babcb323d2b5b => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 12:01:32.522513 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a6212c-278a-4f52-bd93-214735b6a860 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:32.523438 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a6212c-278a-4f52-bd93-214735b6a860 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:32.523438 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a6212c-278a-4f52-bd93-214735b6a860 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:32.575298 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-65a6212c-278a-4f52-bd93-214735b6a860 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:01:32.576852 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-65a6212c-278a-4f52-bd93-214735b6a860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:32.672768 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 915/1838] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:01:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:01:32.681191 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df411bf-5a45-4d97-be7a-4dd1e78f4c52 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:32.682811 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df411bf-5a45-4d97-be7a-4dd1e78f4c52 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:32.682811 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df411bf-5a45-4d97-be7a-4dd1e78f4c52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:32.715709 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-3df411bf-5a45-4d97-be7a-4dd1e78f4c52 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:01:32.717702 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-3df411bf-5a45-4d97-be7a-4dd1e78f4c52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:32.739357 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 924/1839] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:01:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 12:01:32.740950 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a6212c-278a-4f52-bd93-214735b6a860 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:32.759358 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a6212c-278a-4f52-bd93-214735b6a860 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:32.767452 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dd56c9-9fd4-4008-85a7-84f79f6cf648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:32.768088 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dd56c9-9fd4-4008-85a7-84f79f6cf648 admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:32.768369 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dd56c9-9fd4-4008-85a7-84f79f6cf648 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:32.768683 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dd56c9-9fd4-4008-85a7-84f79f6cf648 admin admin] PATH_INFO: `/v3/domains` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:32.769408 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dd56c9-9fd4-4008-85a7-84f79f6cf648 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 12:01:32.769841 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dd56c9-9fd4-4008-85a7-84f79f6cf648 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:32.772570 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dd56c9-9fd4-4008-85a7-84f79f6cf648 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:32.791015 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 916/1840] 10.222.0.59 () {58 vars in 1153 bytes} [Tue Oct 11 12:01:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 12:01:32.797365 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df411bf-5a45-4d97-be7a-4dd1e78f4c52 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:32.817380 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df411bf-5a45-4d97-be7a-4dd1e78f4c52 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:32.831554 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dd56c9-9fd4-4008-85a7-84f79f6cf648 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:32.831836 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29301679-373b-4b85-967c-56c6a9210c11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:32.832971 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29301679-373b-4b85-967c-56c6a9210c11 admin admin] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:32.833433 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29301679-373b-4b85-967c-56c6a9210c11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:32.834160 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29301679-373b-4b85-967c-56c6a9210c11 admin admin] PATH_INFO: `/v3/projects` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:32.835833 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-b7dd56c9-9fd4-4008-85a7-84f79f6cf648 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:32.836643 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29301679-373b-4b85-967c-56c6a9210c11 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:32.841180 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29301679-373b-4b85-967c-56c6a9210c11 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:32.852396 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:01:32.865080 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dd56c9-9fd4-4008-85a7-84f79f6cf648 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:32.876635 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff4df48-7bd8-49dc-beb3-331ea42e2ec6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:32.877714 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff4df48-7bd8-49dc-beb3-331ea42e2ec6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:32.878366 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff4df48-7bd8-49dc-beb3-331ea42e2ec6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:32.878539 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff4df48-7bd8-49dc-beb3-331ea42e2ec6 admin admin] PATH_INFO: `/v3/users/05b7d4e8b9e0435391d9db79e5d68d5a` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:32.880975 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff4df48-7bd8-49dc-beb3-331ea42e2ec6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=05b7d4e8b9e0435391d9db79e5d68d5a)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:32.885175 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff4df48-7bd8-49dc-beb3-331ea42e2ec6 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:32.912661 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 925/1841] 10.222.0.59 () {60 vars in 1153 bytes} [Tue Oct 11 12:01:32 2022] POST /identity/v3/projects => generated 385 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:01:32.915430 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29301679-373b-4b85-967c-56c6a9210c11 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:32.954818 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:01:32.974237 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-2ff4df48-7bd8-49dc-beb3-331ea42e2ec6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05b7d4e8b9e0435391d9db79e5d68d5a'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:33.006113 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-2ff4df48-7bd8-49dc-beb3-331ea42e2ec6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05b7d4e8b9e0435391d9db79e5d68d5a'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:33.021634 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-2ff4df48-7bd8-49dc-beb3-331ea42e2ec6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05b7d4e8b9e0435391d9db79e5d68d5a'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:33.022421 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-2ff4df48-7bd8-49dc-beb3-331ea42e2ec6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05b7d4e8b9e0435391d9db79e5d68d5a'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:33.033345 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-2ff4df48-7bd8-49dc-beb3-331ea42e2ec6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05b7d4e8b9e0435391d9db79e5d68d5a'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:33.042371 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 917/1842] 10.222.0.59 () {58 vars in 1224 bytes} [Tue Oct 11 12:01:32 2022] DELETE /identity/v3/users/05b7d4e8b9e0435391d9db79e5d68d5a => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 12:01:33.048579 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4b5921-3c5e-43c2-b09c-c7023d867dd0 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:33.049156 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4b5921-3c5e-43c2-b09c-c7023d867dd0 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:33.049734 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4b5921-3c5e-43c2-b09c-c7023d867dd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:33.050107 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29301679-373b-4b85-967c-56c6a9210c11 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:33.094985 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-7b4b5921-3c5e-43c2-b09c-c7023d867dd0 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:01:33.096702 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-7b4b5921-3c5e-43c2-b09c-c7023d867dd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:33.167528 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66aba93d-6423-4620-86bd-631af554c306 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:33.168786 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aba93d-6423-4620-86bd-631af554c306 admin admin] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:33.169167 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aba93d-6423-4620-86bd-631af554c306 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:33.169587 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aba93d-6423-4620-86bd-631af554c306 admin admin] PATH_INFO: `/v3/users` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:33.170536 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66aba93d-6423-4620-86bd-631af554c306 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:33.173727 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66aba93d-6423-4620-86bd-631af554c306 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:33.207850 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 926/1843] 10.222.0.59 () {60 vars in 1144 bytes} [Tue Oct 11 12:01:32 2022] POST /identity/v3/users => generated 293 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:01:33.212715 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66aba93d-6423-4620-86bd-631af554c306 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:33.213874 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 918/1844] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:01:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:01:33.216989 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-66aba93d-6423-4620-86bd-631af554c306 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:33.225608 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4b5921-3c5e-43c2-b09c-c7023d867dd0 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:33.227748 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-7b4b5921-3c5e-43c2-b09c-c7023d867dd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:33.237160 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66aba93d-6423-4620-86bd-631af554c306 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:33.244586 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0daba6-4f23-40b6-aa51-022ba41e481e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:33.245783 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0daba6-4f23-40b6-aa51-022ba41e481e admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:33.245962 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0daba6-4f23-40b6-aa51-022ba41e481e admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:33.246240 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0daba6-4f23-40b6-aa51-022ba41e481e admin admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:33.246932 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0daba6-4f23-40b6-aa51-022ba41e481e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:33.249105 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0daba6-4f23-40b6-aa51-022ba41e481e admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:33.271207 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 927/1845] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:01:33 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 11 12:01:33.279382 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0daba6-4f23-40b6-aa51-022ba41e481e admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:33.308948 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0daba6-4f23-40b6-aa51-022ba41e481e admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:33.315237 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f1ebad-1950-468b-8314-08b2a3076591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:33.316268 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f1ebad-1950-468b-8314-08b2a3076591 admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:33.316545 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f1ebad-1950-468b-8314-08b2a3076591 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:33.316996 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f1ebad-1950-468b-8314-08b2a3076591 admin admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:33.317863 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f1ebad-1950-468b-8314-08b2a3076591 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:33.320141 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f1ebad-1950-468b-8314-08b2a3076591 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:33.336691 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 928/1846] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:01:33 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:01:33.337918 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4b5921-3c5e-43c2-b09c-c7023d867dd0 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:33.342434 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f1ebad-1950-468b-8314-08b2a3076591 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:33.347835 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a081dbd8-fcfa-42ed-a568-c79952c520c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e563fe174db4cc19ae6bf8162cc3d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:33.348852 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a081dbd8-fcfa-42ed-a568-c79952c520c3 service neutron] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:33.349534 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a081dbd8-fcfa-42ed-a568-c79952c520c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:33.349993 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a081dbd8-fcfa-42ed-a568-c79952c520c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:33.360563 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f1ebad-1950-468b-8314-08b2a3076591 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:33.370067 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ceb310f-0ad3-421e-93c5-fedea775f21c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:33.371020 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceb310f-0ad3-421e-93c5-fedea775f21c admin admin] REQUEST_METHOD: `PUT` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:33.371612 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceb310f-0ad3-421e-93c5-fedea775f21c admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:33.372145 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceb310f-0ad3-421e-93c5-fedea775f21c admin admin] PATH_INFO: `/v3/projects/cf3edfdac63b433d9685f678621e30f0/users/05738791f4a34b808b2e02f491a7dc76/roles/de2d2bd26fff4748a79990f97a107164` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:33.372429 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a081dbd8-fcfa-42ed-a568-c79952c520c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:33.374544 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a081dbd8-fcfa-42ed-a568-c79952c520c3 service neutron] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:33.393574 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ceb310f-0ad3-421e-93c5-fedea775f21c admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf3edfdac63b433d9685f678621e30f0, user_id=05738791f4a34b808b2e02f491a7dc76, role_id=de2d2bd26fff4748a79990f97a107164)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:33.396483 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 919/1847] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:01:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 11 12:01:33.396656 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ceb310f-0ad3-421e-93c5-fedea775f21c admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:33.421744 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 929/1848] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 12:01:33 2022] PUT /identity/v3/projects/cf3edfdac63b433d9685f678621e30f0/users/05738791f4a34b808b2e02f491a7dc76/roles/de2d2bd26fff4748a79990f97a107164 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:01:33.439966 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5884f0-9be3-4f18-a552-b9ec5577cb7b None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:33.440254 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5884f0-9be3-4f18-a552-b9ec5577cb7b None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:33.440351 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a081dbd8-fcfa-42ed-a568-c79952c520c3 service neutron] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:33.440527 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5884f0-9be3-4f18-a552-b9ec5577cb7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:33.442721 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-a081dbd8-fcfa-42ed-a568-c79952c520c3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:33.469082 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a081dbd8-fcfa-42ed-a568-c79952c520c3 service neutron] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:33.478129 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-ac5884f0-9be3-4f18-a552-b9ec5577cb7b None None] MFA Rules not processed for user `05738791f4a34b808b2e02f491a7dc76`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:01:33.479480 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-ac5884f0-9be3-4f18-a552-b9ec5577cb7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:33.485374 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:01:33.547720 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cf8508-479b-497d-85cd-478a2f324fee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:33.548736 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cf8508-479b-497d-85cd-478a2f324fee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:33.548876 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cf8508-479b-497d-85cd-478a2f324fee admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:33.549088 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cf8508-479b-497d-85cd-478a2f324fee admin admin] PATH_INFO: `/v3/users/de67d44b887c4a868c8bb13343e5f86c` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:33.551465 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cf8508-479b-497d-85cd-478a2f324fee admin admin] RBAC: Authorizing `identity:delete_user(user_id=de67d44b887c4a868c8bb13343e5f86c)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:33.553780 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cf8508-479b-497d-85cd-478a2f324fee admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:33.558188 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:01:33.596858 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 930/1849] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:01:33 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:01:33.611270 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5884f0-9be3-4f18-a552-b9ec5577cb7b None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:33.612815 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-ac5884f0-9be3-4f18-a552-b9ec5577cb7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:33.664876 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-34cf8508-479b-497d-85cd-478a2f324fee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de67d44b887c4a868c8bb13343e5f86c'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:33.684911 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-34cf8508-479b-497d-85cd-478a2f324fee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de67d44b887c4a868c8bb13343e5f86c'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:33.701159 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-34cf8508-479b-497d-85cd-478a2f324fee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de67d44b887c4a868c8bb13343e5f86c'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:33.701885 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-34cf8508-479b-497d-85cd-478a2f324fee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de67d44b887c4a868c8bb13343e5f86c'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:33.725032 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-34cf8508-479b-497d-85cd-478a2f324fee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de67d44b887c4a868c8bb13343e5f86c'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:33.748495 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 920/1850] 10.222.0.59 () {58 vars in 1224 bytes} [Tue Oct 11 12:01:33 2022] DELETE /identity/v3/users/de67d44b887c4a868c8bb13343e5f86c => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:01:33.754104 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7928560-d6b9-42b0-a982-67c8a79b354b None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:33.754355 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5884f0-9be3-4f18-a552-b9ec5577cb7b None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:33.754892 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7928560-d6b9-42b0-a982-67c8a79b354b None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:33.755120 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7928560-d6b9-42b0-a982-67c8a79b354b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:33.805387 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-c7928560-d6b9-42b0-a982-67c8a79b354b None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:01:33.806843 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-c7928560-d6b9-42b0-a982-67c8a79b354b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:33.872855 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fba0a3e-2355-406d-a2d7-a6c242066b35 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b1b651c73664d08accc6edf2f79bace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:33.873521 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fba0a3e-2355-406d-a2d7-a6c242066b35 service cinder] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:33.873821 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fba0a3e-2355-406d-a2d7-a6c242066b35 service cinder] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:33.874080 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fba0a3e-2355-406d-a2d7-a6c242066b35 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:33.876254 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-9fba0a3e-2355-406d-a2d7-a6c242066b35 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:33.926770 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 921/1851] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:01:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:01:33.929373 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7928560-d6b9-42b0-a982-67c8a79b354b None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:33.932665 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-c7928560-d6b9-42b0-a982-67c8a79b354b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:33.950708 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7928560-d6b9-42b0-a982-67c8a79b354b None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:33.957904 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37977b37-e578-4d3d-b65b-330fe0e025f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:33.958669 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37977b37-e578-4d3d-b65b-330fe0e025f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:33.959095 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37977b37-e578-4d3d-b65b-330fe0e025f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:33.959355 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37977b37-e578-4d3d-b65b-330fe0e025f6 admin admin] PATH_INFO: `/v3/projects/4fef4ffc374a4e72bfffbb9e1d6411fa` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:33.964021 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37977b37-e578-4d3d-b65b-330fe0e025f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4fef4ffc374a4e72bfffbb9e1d6411fa)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:33.966224 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37977b37-e578-4d3d-b65b-330fe0e025f6 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:34.008282 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fba0a3e-2355-406d-a2d7-a6c242066b35 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:34.012402 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fba0a3e-2355-406d-a2d7-a6c242066b35 service cinder] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:34.031610 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:01:34.038255 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 931/1852] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:01:33 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 11 12:01:34.040032 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fba0a3e-2355-406d-a2d7-a6c242066b35 service cinder] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:34.042379 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-9fba0a3e-2355-406d-a2d7-a6c242066b35 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:34.082294 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-37977b37-e578-4d3d-b65b-330fe0e025f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4fef4ffc374a4e72bfffbb9e1d6411fa', 'request_id': 'req-37977b37-e578-4d3d-b65b-330fe0e025f6'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:34.099922 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-37977b37-e578-4d3d-b65b-330fe0e025f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4fef4ffc374a4e72bfffbb9e1d6411fa', 'request_id': 'req-37977b37-e578-4d3d-b65b-330fe0e025f6'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:34.116985 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-37977b37-e578-4d3d-b65b-330fe0e025f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4fef4ffc374a4e72bfffbb9e1d6411fa', 'request_id': 'req-37977b37-e578-4d3d-b65b-330fe0e025f6'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:34.140313 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-37977b37-e578-4d3d-b65b-330fe0e025f6 admin admin] The token cache is being invalidate because project 4fef4ffc374a4e72bfffbb9e1d6411fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 12:01:34.140989 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-37977b37-e578-4d3d-b65b-330fe0e025f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:34.142618 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-37977b37-e578-4d3d-b65b-330fe0e025f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:34.150160 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 922/1853] 10.222.0.59 () {58 vars in 1233 bytes} [Tue Oct 11 12:01:33 2022] DELETE /identity/v3/projects/4fef4ffc374a4e72bfffbb9e1d6411fa => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:01:34.152487 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37977b37-e578-4d3d-b65b-330fe0e025f6 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:34.155303 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-37977b37-e578-4d3d-b65b-330fe0e025f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:34.190776 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fba0a3e-2355-406d-a2d7-a6c242066b35 service cinder] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:34.227714 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:01:34.289674 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37977b37-e578-4d3d-b65b-330fe0e025f6 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:34.302382 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151599d3-ae1a-4352-afb1-fbf646ff8f0d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e563fe174db4cc19ae6bf8162cc3d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:34.303710 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151599d3-ae1a-4352-afb1-fbf646ff8f0d service neutron] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:34.303710 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151599d3-ae1a-4352-afb1-fbf646ff8f0d service neutron] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:34.303710 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151599d3-ae1a-4352-afb1-fbf646ff8f0d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:34.306649 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-151599d3-ae1a-4352-afb1-fbf646ff8f0d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:34.311139 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3b4b70-6ba0-42d9-b858-9f402d18d1a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:34.314968 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3b4b70-6ba0-42d9-b858-9f402d18d1a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:34.315798 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3b4b70-6ba0-42d9-b858-9f402d18d1a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:34.316731 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3b4b70-6ba0-42d9-b858-9f402d18d1a2 admin admin] PATH_INFO: `/v3/users/5b4cbb0e3df44c1aab3bb90ebf6cf8c1` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:34.320174 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3b4b70-6ba0-42d9-b858-9f402d18d1a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b4cbb0e3df44c1aab3bb90ebf6cf8c1)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:34.325987 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151599d3-ae1a-4352-afb1-fbf646ff8f0d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:34.327070 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3b4b70-6ba0-42d9-b858-9f402d18d1a2 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:34.327556 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151599d3-ae1a-4352-afb1-fbf646ff8f0d service neutron] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:34.341081 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 932/1854] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:01:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:01:34.439224 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-2b3b4b70-6ba0-42d9-b858-9f402d18d1a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b4cbb0e3df44c1aab3bb90ebf6cf8c1'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:34.481928 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-2b3b4b70-6ba0-42d9-b858-9f402d18d1a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b4cbb0e3df44c1aab3bb90ebf6cf8c1'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:34.499833 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-2b3b4b70-6ba0-42d9-b858-9f402d18d1a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b4cbb0e3df44c1aab3bb90ebf6cf8c1'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:34.501711 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-2b3b4b70-6ba0-42d9-b858-9f402d18d1a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b4cbb0e3df44c1aab3bb90ebf6cf8c1'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:34.524266 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-2b3b4b70-6ba0-42d9-b858-9f402d18d1a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b4cbb0e3df44c1aab3bb90ebf6cf8c1'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:34.537823 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 923/1855] 10.222.0.59 () {58 vars in 1224 bytes} [Tue Oct 11 12:01:34 2022] DELETE /identity/v3/users/5b4cbb0e3df44c1aab3bb90ebf6cf8c1 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:01:34.833625 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151599d3-ae1a-4352-afb1-fbf646ff8f0d service neutron] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:34.836329 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-151599d3-ae1a-4352-afb1-fbf646ff8f0d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:34.879053 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151599d3-ae1a-4352-afb1-fbf646ff8f0d service neutron] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:34.966030 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e231ab-d1a6-4296-a81e-549a2426c9fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:34.966791 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e231ab-d1a6-4296-a81e-549a2426c9fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:34.967128 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e231ab-d1a6-4296-a81e-549a2426c9fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:34.967414 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e231ab-d1a6-4296-a81e-549a2426c9fc admin admin] PATH_INFO: `/v3/projects/48276d0b291e47038b0cbabd5b68952a` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:34.968805 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e231ab-d1a6-4296-a81e-549a2426c9fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=48276d0b291e47038b0cbabd5b68952a)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:34.971315 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e231ab-d1a6-4296-a81e-549a2426c9fc admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:34.981841 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3b4b70-6ba0-42d9-b858-9f402d18d1a2 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:34.984290 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-2b3b4b70-6ba0-42d9-b858-9f402d18d1a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:35.000490 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3b4b70-6ba0-42d9-b858-9f402d18d1a2 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:35.009637 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c415ee67-e9ac-4e8c-a52e-dc602592da43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:35.010893 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c415ee67-e9ac-4e8c-a52e-dc602592da43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:35.011399 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c415ee67-e9ac-4e8c-a52e-dc602592da43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:35.011869 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c415ee67-e9ac-4e8c-a52e-dc602592da43 admin admin] PATH_INFO: `/v3/projects/80b50c7217d740778d256b877daee1d0` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:35.013788 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c415ee67-e9ac-4e8c-a52e-dc602592da43 admin admin] RBAC: Authorizing `identity:delete_project(project_id=80b50c7217d740778d256b877daee1d0)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:35.017644 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c415ee67-e9ac-4e8c-a52e-dc602592da43 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:35.069163 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:01:35.071517 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-92e231ab-d1a6-4296-a81e-549a2426c9fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48276d0b291e47038b0cbabd5b68952a', 'request_id': 'req-92e231ab-d1a6-4296-a81e-549a2426c9fc'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:35.081649 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-c415ee67-e9ac-4e8c-a52e-dc602592da43 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80b50c7217d740778d256b877daee1d0', 'request_id': 'req-c415ee67-e9ac-4e8c-a52e-dc602592da43'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:35.087512 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-92e231ab-d1a6-4296-a81e-549a2426c9fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48276d0b291e47038b0cbabd5b68952a', 'request_id': 'req-92e231ab-d1a6-4296-a81e-549a2426c9fc'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:35.089845 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-c415ee67-e9ac-4e8c-a52e-dc602592da43 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80b50c7217d740778d256b877daee1d0', 'request_id': 'req-c415ee67-e9ac-4e8c-a52e-dc602592da43'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:35.097051 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-92e231ab-d1a6-4296-a81e-549a2426c9fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48276d0b291e47038b0cbabd5b68952a', 'request_id': 'req-92e231ab-d1a6-4296-a81e-549a2426c9fc'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:35.099391 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-c415ee67-e9ac-4e8c-a52e-dc602592da43 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80b50c7217d740778d256b877daee1d0', 'request_id': 'req-c415ee67-e9ac-4e8c-a52e-dc602592da43'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:35.113523 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-92e231ab-d1a6-4296-a81e-549a2426c9fc admin admin] The token cache is being invalidate because project 48276d0b291e47038b0cbabd5b68952a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81432) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 12:01:35.113849 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-92e231ab-d1a6-4296-a81e-549a2426c9fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:35.114952 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-92e231ab-d1a6-4296-a81e-549a2426c9fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:35.116941 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 933/1856] 10.222.0.59 () {58 vars in 1233 bytes} [Tue Oct 11 12:01:34 2022] DELETE /identity/v3/projects/48276d0b291e47038b0cbabd5b68952a => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:01:35.119884 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-c415ee67-e9ac-4e8c-a52e-dc602592da43 admin admin] The token cache is being invalidate because project 80b50c7217d740778d256b877daee1d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 12:01:35.120387 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-c415ee67-e9ac-4e8c-a52e-dc602592da43 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:35.121403 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-c415ee67-e9ac-4e8c-a52e-dc602592da43 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:35.123948 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 924/1857] 10.222.0.59 () {58 vars in 1233 bytes} [Tue Oct 11 12:01:34 2022] DELETE /identity/v3/projects/80b50c7217d740778d256b877daee1d0 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:01:35.124303 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e231ab-d1a6-4296-a81e-549a2426c9fc admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:35.125942 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-92e231ab-d1a6-4296-a81e-549a2426c9fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:35.128999 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c415ee67-e9ac-4e8c-a52e-dc602592da43 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:35.131339 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-c415ee67-e9ac-4e8c-a52e-dc602592da43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:35.210393 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:01:35.224724 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e231ab-d1a6-4296-a81e-549a2426c9fc admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:35.235976 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e254bb8b-a47c-4c35-a244-f8935befee78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:35.236916 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c415ee67-e9ac-4e8c-a52e-dc602592da43 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:35.237357 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e254bb8b-a47c-4c35-a244-f8935befee78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:35.237918 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e254bb8b-a47c-4c35-a244-f8935befee78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:35.238403 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e254bb8b-a47c-4c35-a244-f8935befee78 admin admin] PATH_INFO: `/v3/users/5cfb1872b5384903bbe48686a1a58879` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:35.240181 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e254bb8b-a47c-4c35-a244-f8935befee78 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5cfb1872b5384903bbe48686a1a58879)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:35.242825 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e254bb8b-a47c-4c35-a244-f8935befee78 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:35.243036 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ae7f1b-501a-4e07-8d6f-829aad3bbcf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:35.243660 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ae7f1b-501a-4e07-8d6f-829aad3bbcf3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:35.243927 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ae7f1b-501a-4e07-8d6f-829aad3bbcf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:35.244173 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ae7f1b-501a-4e07-8d6f-829aad3bbcf3 admin admin] PATH_INFO: `/v3/users/c476c7c73fde4fdca1a27f0e0bbc3499` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:35.246829 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ae7f1b-501a-4e07-8d6f-829aad3bbcf3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c476c7c73fde4fdca1a27f0e0bbc3499)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:35.248010 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ae7f1b-501a-4e07-8d6f-829aad3bbcf3 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:35.325541 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-53ae7f1b-501a-4e07-8d6f-829aad3bbcf3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c476c7c73fde4fdca1a27f0e0bbc3499'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:35.327902 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-e254bb8b-a47c-4c35-a244-f8935befee78 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5cfb1872b5384903bbe48686a1a58879'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:35.348209 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-53ae7f1b-501a-4e07-8d6f-829aad3bbcf3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c476c7c73fde4fdca1a27f0e0bbc3499'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:35.354638 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-e254bb8b-a47c-4c35-a244-f8935befee78 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5cfb1872b5384903bbe48686a1a58879'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:35.355010 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-e254bb8b-a47c-4c35-a244-f8935befee78 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5cfb1872b5384903bbe48686a1a58879'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:35.363931 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-53ae7f1b-501a-4e07-8d6f-829aad3bbcf3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c476c7c73fde4fdca1a27f0e0bbc3499'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:35.364817 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-53ae7f1b-501a-4e07-8d6f-829aad3bbcf3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c476c7c73fde4fdca1a27f0e0bbc3499'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:35.367573 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-e254bb8b-a47c-4c35-a244-f8935befee78 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5cfb1872b5384903bbe48686a1a58879'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:35.368455 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-e254bb8b-a47c-4c35-a244-f8935befee78 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5cfb1872b5384903bbe48686a1a58879'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:35.379818 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-53ae7f1b-501a-4e07-8d6f-829aad3bbcf3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c476c7c73fde4fdca1a27f0e0bbc3499'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:35.389945 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 934/1859] 10.222.0.59 () {58 vars in 1224 bytes} [Tue Oct 11 12:01:35 2022] DELETE /identity/v3/users/5cfb1872b5384903bbe48686a1a58879 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 12:01:35.390248 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 925/1859] 10.222.0.59 () {58 vars in 1224 bytes} [Tue Oct 11 12:01:35 2022] DELETE /identity/v3/users/c476c7c73fde4fdca1a27f0e0bbc3499 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 12:01:35.827035 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ae7f1b-501a-4e07-8d6f-829aad3bbcf3 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:35.828885 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-53ae7f1b-501a-4e07-8d6f-829aad3bbcf3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:35.841414 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e254bb8b-a47c-4c35-a244-f8935befee78 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:35.843339 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-e254bb8b-a47c-4c35-a244-f8935befee78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:35.893294 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:01:35.919466 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ae7f1b-501a-4e07-8d6f-829aad3bbcf3 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:35.929933 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caf7fb3-3268-4fc0-a4f7-a5078253d94a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:35.930826 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caf7fb3-3268-4fc0-a4f7-a5078253d94a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:35.931185 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caf7fb3-3268-4fc0-a4f7-a5078253d94a admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:35.931531 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caf7fb3-3268-4fc0-a4f7-a5078253d94a admin admin] PATH_INFO: `/v3/projects/583873d675d24d0b942029f1614dd31d` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:35.932927 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6caf7fb3-3268-4fc0-a4f7-a5078253d94a admin admin] RBAC: Authorizing `identity:delete_project(project_id=583873d675d24d0b942029f1614dd31d)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:35.935138 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6caf7fb3-3268-4fc0-a4f7-a5078253d94a admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:35.976732 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e254bb8b-a47c-4c35-a244-f8935befee78 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:35.983094 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dabe78-5bf2-45ff-9ef4-63b3ab2beaf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:35.983761 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dabe78-5bf2-45ff-9ef4-63b3ab2beaf8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:35.984076 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dabe78-5bf2-45ff-9ef4-63b3ab2beaf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:35.984342 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dabe78-5bf2-45ff-9ef4-63b3ab2beaf8 admin admin] PATH_INFO: `/v3/projects/d8214d1e985a4188b701f104e45f6805` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:35.985583 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dabe78-5bf2-45ff-9ef4-63b3ab2beaf8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8214d1e985a4188b701f104e45f6805)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:35.987618 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dabe78-5bf2-45ff-9ef4-63b3ab2beaf8 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:36.003545 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-6caf7fb3-3268-4fc0-a4f7-a5078253d94a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '583873d675d24d0b942029f1614dd31d', 'request_id': 'req-6caf7fb3-3268-4fc0-a4f7-a5078253d94a'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:36.014526 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-6caf7fb3-3268-4fc0-a4f7-a5078253d94a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '583873d675d24d0b942029f1614dd31d', 'request_id': 'req-6caf7fb3-3268-4fc0-a4f7-a5078253d94a'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:36.031401 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-6caf7fb3-3268-4fc0-a4f7-a5078253d94a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '583873d675d24d0b942029f1614dd31d', 'request_id': 'req-6caf7fb3-3268-4fc0-a4f7-a5078253d94a'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:36.054101 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-6caf7fb3-3268-4fc0-a4f7-a5078253d94a admin admin] The token cache is being invalidate because project 583873d675d24d0b942029f1614dd31d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 12:01:36.054675 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-6caf7fb3-3268-4fc0-a4f7-a5078253d94a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:36.055776 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-6caf7fb3-3268-4fc0-a4f7-a5078253d94a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:36.059747 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-50dabe78-5bf2-45ff-9ef4-63b3ab2beaf8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8214d1e985a4188b701f104e45f6805', 'request_id': 'req-50dabe78-5bf2-45ff-9ef4-63b3ab2beaf8'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:36.063772 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 926/1860] 10.222.0.59 () {58 vars in 1233 bytes} [Tue Oct 11 12:01:35 2022] DELETE /identity/v3/projects/583873d675d24d0b942029f1614dd31d => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:01:36.066497 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefcdc48-9bbd-4330-a5e1-ff21fad79617 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:36.066731 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefcdc48-9bbd-4330-a5e1-ff21fad79617 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:36.067100 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-50dabe78-5bf2-45ff-9ef4-63b3ab2beaf8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8214d1e985a4188b701f104e45f6805', 'request_id': 'req-50dabe78-5bf2-45ff-9ef4-63b3ab2beaf8'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:36.067224 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefcdc48-9bbd-4330-a5e1-ff21fad79617 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:36.074826 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-50dabe78-5bf2-45ff-9ef4-63b3ab2beaf8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8214d1e985a4188b701f104e45f6805', 'request_id': 'req-50dabe78-5bf2-45ff-9ef4-63b3ab2beaf8'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:36.093662 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-50dabe78-5bf2-45ff-9ef4-63b3ab2beaf8 admin admin] The token cache is being invalidate because project d8214d1e985a4188b701f104e45f6805 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81432) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 12:01:36.094012 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-50dabe78-5bf2-45ff-9ef4-63b3ab2beaf8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:36.094928 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-50dabe78-5bf2-45ff-9ef4-63b3ab2beaf8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:36.096660 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 935/1861] 10.222.0.59 () {58 vars in 1233 bytes} [Tue Oct 11 12:01:35 2022] DELETE /identity/v3/projects/d8214d1e985a4188b701f104e45f6805 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 12:01:36.102206 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc37e52-ebcf-4d16-80a3-109a47bdcd30 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:36.102667 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc37e52-ebcf-4d16-80a3-109a47bdcd30 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:36.102967 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc37e52-ebcf-4d16-80a3-109a47bdcd30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:36.112229 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:01:36.114067 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-fefcdc48-9bbd-4330-a5e1-ff21fad79617 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:01:36.115654 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-fefcdc48-9bbd-4330-a5e1-ff21fad79617 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:36.133844 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-1fc37e52-ebcf-4d16-80a3-109a47bdcd30 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:01:36.135195 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-1fc37e52-ebcf-4d16-80a3-109a47bdcd30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:36.215675 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 936/1862] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:01:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 12:01:36.223557 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 927/1863] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:01:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:01:36.230006 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c0a080-9297-42fd-b522-afb93fdde9e5 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:36.230006 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c0a080-9297-42fd-b522-afb93fdde9e5 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:36.230006 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c0a080-9297-42fd-b522-afb93fdde9e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:36.233629 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6de5faf-e9eb-4e58-a55b-0adef927b6ce None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:36.233920 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6de5faf-e9eb-4e58-a55b-0adef927b6ce None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:36.234333 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6de5faf-e9eb-4e58-a55b-0adef927b6ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:36.255571 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-c6de5faf-e9eb-4e58-a55b-0adef927b6ce None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:01:36.256805 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-c6de5faf-e9eb-4e58-a55b-0adef927b6ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:36.257997 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-32c0a080-9297-42fd-b522-afb93fdde9e5 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:01:36.260074 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-32c0a080-9297-42fd-b522-afb93fdde9e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:36.270497 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 928/1864] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:01:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 12:01:36.274801 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6de5faf-e9eb-4e58-a55b-0adef927b6ce None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:36.275212 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 937/1865] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:01:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:01:36.281578 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c0a080-9297-42fd-b522-afb93fdde9e5 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:36.285636 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6de5faf-e9eb-4e58-a55b-0adef927b6ce None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:36.291283 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c0a080-9297-42fd-b522-afb93fdde9e5 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:36.291787 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc59cc6a-2317-4d9d-a9f9-ab209f8c9d77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:36.292425 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc59cc6a-2317-4d9d-a9f9-ab209f8c9d77 admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:36.292769 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc59cc6a-2317-4d9d-a9f9-ab209f8c9d77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:36.292984 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc59cc6a-2317-4d9d-a9f9-ab209f8c9d77 admin admin] PATH_INFO: `/v3/domains` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:36.293651 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc59cc6a-2317-4d9d-a9f9-ab209f8c9d77 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 12:01:36.293955 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc59cc6a-2317-4d9d-a9f9-ab209f8c9d77 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:36.295986 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc59cc6a-2317-4d9d-a9f9-ab209f8c9d77 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:36.299444 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2a2591-5415-433a-8abd-007c79b1c433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:36.300750 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2a2591-5415-433a-8abd-007c79b1c433 admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:36.301301 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2a2591-5415-433a-8abd-007c79b1c433 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:36.301804 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2a2591-5415-433a-8abd-007c79b1c433 admin admin] PATH_INFO: `/v3/domains` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:36.302832 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2a2591-5415-433a-8abd-007c79b1c433 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 12:01:36.303317 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2a2591-5415-433a-8abd-007c79b1c433 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:36.305817 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2a2591-5415-433a-8abd-007c79b1c433 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:36.313188 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 929/1866] 10.222.0.59 () {58 vars in 1153 bytes} [Tue Oct 11 12:01:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 12:01:36.316712 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc59cc6a-2317-4d9d-a9f9-ab209f8c9d77 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:36.323864 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 938/1867] 10.222.0.59 () {58 vars in 1153 bytes} [Tue Oct 11 12:01:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 12:01:36.327912 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc59cc6a-2317-4d9d-a9f9-ab209f8c9d77 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:36.329060 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2a2591-5415-433a-8abd-007c79b1c433 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:36.333516 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80eeeb5-6850-4363-95e7-3cb882771e40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:36.334218 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80eeeb5-6850-4363-95e7-3cb882771e40 admin admin] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:36.334503 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80eeeb5-6850-4363-95e7-3cb882771e40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:36.334820 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80eeeb5-6850-4363-95e7-3cb882771e40 admin admin] PATH_INFO: `/v3/projects` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:36.335512 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80eeeb5-6850-4363-95e7-3cb882771e40 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:36.337441 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80eeeb5-6850-4363-95e7-3cb882771e40 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:36.344095 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2a2591-5415-433a-8abd-007c79b1c433 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:36.351394 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c915d3-634b-40cd-9def-488e948d0d0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:36.352454 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c915d3-634b-40cd-9def-488e948d0d0a admin admin] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:36.352809 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c915d3-634b-40cd-9def-488e948d0d0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:36.353292 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c915d3-634b-40cd-9def-488e948d0d0a admin admin] PATH_INFO: `/v3/projects` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:36.354495 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c915d3-634b-40cd-9def-488e948d0d0a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:36.358235 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c915d3-634b-40cd-9def-488e948d0d0a admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:36.372487 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 930/1868] 10.222.0.59 () {60 vars in 1153 bytes} [Tue Oct 11 12:01:36 2022] POST /identity/v3/projects => generated 385 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 12:01:36.373568 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80eeeb5-6850-4363-95e7-3cb882771e40 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:36.412031 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 939/1869] 10.222.0.59 () {60 vars in 1153 bytes} [Tue Oct 11 12:01:36 2022] POST /identity/v3/projects => generated 371 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 12:01:36.416907 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c915d3-634b-40cd-9def-488e948d0d0a admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:36.443586 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:01:36.496251 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80eeeb5-6850-4363-95e7-3cb882771e40 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:36.502010 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea409b55-774e-40f4-ad34-c2bd4619b74c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:36.502799 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea409b55-774e-40f4-ad34-c2bd4619b74c admin admin] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:36.503145 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea409b55-774e-40f4-ad34-c2bd4619b74c admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:36.503490 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea409b55-774e-40f4-ad34-c2bd4619b74c admin admin] PATH_INFO: `/v3/users` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:36.504219 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea409b55-774e-40f4-ad34-c2bd4619b74c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:36.506267 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea409b55-774e-40f4-ad34-c2bd4619b74c admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:36.510822 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c915d3-634b-40cd-9def-488e948d0d0a admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:36.519338 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d4ddb9-ac53-45ea-afb8-846627150481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:36.520509 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d4ddb9-ac53-45ea-afb8-846627150481 admin admin] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:36.521106 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d4ddb9-ac53-45ea-afb8-846627150481 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:36.521725 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d4ddb9-ac53-45ea-afb8-846627150481 admin admin] PATH_INFO: `/v3/users` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:36.523343 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d4ddb9-ac53-45ea-afb8-846627150481 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:36.527818 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d4ddb9-ac53-45ea-afb8-846627150481 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:36.536257 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 931/1870] 10.222.0.59 () {60 vars in 1144 bytes} [Tue Oct 11 12:01:36 2022] POST /identity/v3/users => generated 293 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 12:01:36.543276 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea409b55-774e-40f4-ad34-c2bd4619b74c admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:36.554396 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea409b55-774e-40f4-ad34-c2bd4619b74c admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:36.565222 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3b70a8-2430-47af-b9a3-a31a4d1a57c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:36.565905 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3b70a8-2430-47af-b9a3-a31a4d1a57c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:36.566166 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3b70a8-2430-47af-b9a3-a31a4d1a57c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:36.567447 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3b70a8-2430-47af-b9a3-a31a4d1a57c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:36.568094 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3b70a8-2430-47af-b9a3-a31a4d1a57c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:36.569839 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3b70a8-2430-47af-b9a3-a31a4d1a57c4 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:36.578999 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 940/1871] 10.222.0.59 () {60 vars in 1144 bytes} [Tue Oct 11 12:01:36 2022] POST /identity/v3/users => generated 286 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:01:36.583269 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d4ddb9-ac53-45ea-afb8-846627150481 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:36.592792 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 932/1872] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:01:36 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 11 12:01:36.594440 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d4ddb9-ac53-45ea-afb8-846627150481 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:36.597198 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3b70a8-2430-47af-b9a3-a31a4d1a57c4 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:36.600467 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262790b7-f046-4379-80b1-fdbf1b670d41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:36.601118 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262790b7-f046-4379-80b1-fdbf1b670d41 admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:36.601461 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262790b7-f046-4379-80b1-fdbf1b670d41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:36.601745 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262790b7-f046-4379-80b1-fdbf1b670d41 admin admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:36.602620 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262790b7-f046-4379-80b1-fdbf1b670d41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:36.604436 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262790b7-f046-4379-80b1-fdbf1b670d41 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:36.608942 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3b70a8-2430-47af-b9a3-a31a4d1a57c4 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:36.614695 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d271bbb-1b3e-491b-901f-525f1c718411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:36.615435 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d271bbb-1b3e-491b-901f-525f1c718411 admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:36.615845 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d271bbb-1b3e-491b-901f-525f1c718411 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:36.616201 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d271bbb-1b3e-491b-901f-525f1c718411 admin admin] PATH_INFO: `/v3/roles` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:36.617081 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d271bbb-1b3e-491b-901f-525f1c718411 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:36.624821 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 941/1873] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:01:36 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 11 12:01:36.625746 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d271bbb-1b3e-491b-901f-525f1c718411 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:36.632826 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262790b7-f046-4379-80b1-fdbf1b670d41 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:36.644108 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262790b7-f046-4379-80b1-fdbf1b670d41 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:36.647954 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 933/1874] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:01:36 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 11 12:01:36.653569 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d271bbb-1b3e-491b-901f-525f1c718411 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:36.655338 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd9cc23-7412-41aa-a3d1-6eedca7cd5b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:36.656723 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd9cc23-7412-41aa-a3d1-6eedca7cd5b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:36.662119 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd9cc23-7412-41aa-a3d1-6eedca7cd5b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:36.662472 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd9cc23-7412-41aa-a3d1-6eedca7cd5b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:36.663477 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd9cc23-7412-41aa-a3d1-6eedca7cd5b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:36.665177 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd9cc23-7412-41aa-a3d1-6eedca7cd5b2 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:36.673008 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d271bbb-1b3e-491b-901f-525f1c718411 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:36.677607 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 942/1875] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:01:36 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 11 12:01:36.682483 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd9cc23-7412-41aa-a3d1-6eedca7cd5b2 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:36.682730 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f5b8c9-1659-4235-9c36-326214f58572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:36.683706 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f5b8c9-1659-4235-9c36-326214f58572 admin admin] REQUEST_METHOD: `PUT` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:36.684183 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f5b8c9-1659-4235-9c36-326214f58572 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:36.684736 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f5b8c9-1659-4235-9c36-326214f58572 admin admin] PATH_INFO: `/v3/projects/b4fe478e16b2424c96ad943e4a165e54/users/c4ec79b8301748e5bfa179a5c96df1ee/roles/de2d2bd26fff4748a79990f97a107164` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:36.704168 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd9cc23-7412-41aa-a3d1-6eedca7cd5b2 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:36.713186 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ca58d7-200a-4b7a-b2b3-9f344a89d909 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:36.714718 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ca58d7-200a-4b7a-b2b3-9f344a89d909 admin admin] REQUEST_METHOD: `PUT` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:36.715286 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ca58d7-200a-4b7a-b2b3-9f344a89d909 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:36.715825 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ca58d7-200a-4b7a-b2b3-9f344a89d909 admin admin] PATH_INFO: `/v3/projects/f943e2a175344e40b29c72019ba95b03/users/ae372b81003e41aeb654da0bcca928d5/roles/de2d2bd26fff4748a79990f97a107164` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:36.715937 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f5b8c9-1659-4235-9c36-326214f58572 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4fe478e16b2424c96ad943e4a165e54, user_id=c4ec79b8301748e5bfa179a5c96df1ee, role_id=de2d2bd26fff4748a79990f97a107164)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:36.723008 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f5b8c9-1659-4235-9c36-326214f58572 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:36.750625 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ca58d7-200a-4b7a-b2b3-9f344a89d909 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f943e2a175344e40b29c72019ba95b03, user_id=ae372b81003e41aeb654da0bcca928d5, role_id=de2d2bd26fff4748a79990f97a107164)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:36.753661 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 934/1876] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 12:01:36 2022] PUT /identity/v3/projects/b4fe478e16b2424c96ad943e4a165e54/users/c4ec79b8301748e5bfa179a5c96df1ee/roles/de2d2bd26fff4748a79990f97a107164 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 12:01:36.756340 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ca58d7-200a-4b7a-b2b3-9f344a89d909 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:36.764682 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186f068a-03a0-4040-95f4-2a9bb4ec4b4c None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:36.765003 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186f068a-03a0-4040-95f4-2a9bb4ec4b4c None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:36.765346 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186f068a-03a0-4040-95f4-2a9bb4ec4b4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:36.778386 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 943/1877] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 12:01:36 2022] PUT /identity/v3/projects/f943e2a175344e40b29c72019ba95b03/users/ae372b81003e41aeb654da0bcca928d5/roles/de2d2bd26fff4748a79990f97a107164 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 12:01:36.785232 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db02a9b-0e12-4608-ad10-72815dac1376 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:36.785642 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db02a9b-0e12-4608-ad10-72815dac1376 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:36.786027 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db02a9b-0e12-4608-ad10-72815dac1376 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:36.795932 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:01:36.804689 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-186f068a-03a0-4040-95f4-2a9bb4ec4b4c None None] MFA Rules not processed for user `c4ec79b8301748e5bfa179a5c96df1ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:01:36.804689 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-186f068a-03a0-4040-95f4-2a9bb4ec4b4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:36.811269 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-7db02a9b-0e12-4608-ad10-72815dac1376 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:01:36.812884 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-7db02a9b-0e12-4608-ad10-72815dac1376 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:36.882023 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 935/1878] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:01:36 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 12:01:36.886902 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 944/1879] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:01:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 12:01:36.895602 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186f068a-03a0-4040-95f4-2a9bb4ec4b4c None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:36.897204 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db02a9b-0e12-4608-ad10-72815dac1376 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:36.897330 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-186f068a-03a0-4040-95f4-2a9bb4ec4b4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:36.899648 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-7db02a9b-0e12-4608-ad10-72815dac1376 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:36.973059 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186f068a-03a0-4040-95f4-2a9bb4ec4b4c None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:36.977209 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9f71f8-3349-4817-990d-8706236e7be3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b1b651c73664d08accc6edf2f79bace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:36.977850 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9f71f8-3349-4817-990d-8706236e7be3 service cinder] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:36.978130 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9f71f8-3349-4817-990d-8706236e7be3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:36.978404 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9f71f8-3349-4817-990d-8706236e7be3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:36.989807 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9f71f8-3349-4817-990d-8706236e7be3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:36.992389 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9f71f8-3349-4817-990d-8706236e7be3 service cinder] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:36.992570 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db02a9b-0e12-4608-ad10-72815dac1376 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:36.996354 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9040b510-0135-4a0c-90a8-5622dda3166c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e563fe174db4cc19ae6bf8162cc3d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:36.996993 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9040b510-0135-4a0c-90a8-5622dda3166c service neutron] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:36.997275 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9040b510-0135-4a0c-90a8-5622dda3166c service neutron] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:36.997554 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9040b510-0135-4a0c-90a8-5622dda3166c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:37.003939 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 936/1880] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:01:36 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:01:37.009661 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9040b510-0135-4a0c-90a8-5622dda3166c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:37.012342 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9040b510-0135-4a0c-90a8-5622dda3166c service neutron] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:37.025244 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9f71f8-3349-4817-990d-8706236e7be3 service cinder] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:37.027577 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 945/1881] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:01:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:01:37.038174 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9f71f8-3349-4817-990d-8706236e7be3 service cinder] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:37.044106 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806753ef-1075-48a5-b298-3a1501c454b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:37.045233 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806753ef-1075-48a5-b298-3a1501c454b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:37.045693 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806753ef-1075-48a5-b298-3a1501c454b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:37.045976 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806753ef-1075-48a5-b298-3a1501c454b6 admin admin] PATH_INFO: `/v3/users/c4ec79b8301748e5bfa179a5c96df1ee` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:37.048089 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806753ef-1075-48a5-b298-3a1501c454b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c4ec79b8301748e5bfa179a5c96df1ee)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:37.050316 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806753ef-1075-48a5-b298-3a1501c454b6 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:37.052312 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:01:37.113385 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-806753ef-1075-48a5-b298-3a1501c454b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c4ec79b8301748e5bfa179a5c96df1ee'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:37.137143 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-806753ef-1075-48a5-b298-3a1501c454b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c4ec79b8301748e5bfa179a5c96df1ee'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:37.150040 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-806753ef-1075-48a5-b298-3a1501c454b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c4ec79b8301748e5bfa179a5c96df1ee'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:37.151001 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-806753ef-1075-48a5-b298-3a1501c454b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c4ec79b8301748e5bfa179a5c96df1ee'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:37.168859 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-806753ef-1075-48a5-b298-3a1501c454b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c4ec79b8301748e5bfa179a5c96df1ee'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:37.180589 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 937/1882] 10.222.0.59 () {58 vars in 1224 bytes} [Tue Oct 11 12:01:37 2022] DELETE /identity/v3/users/c4ec79b8301748e5bfa179a5c96df1ee => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:01:37.186617 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103398ca-1982-4d49-a2e1-d28ae98b526a None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:37.186896 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103398ca-1982-4d49-a2e1-d28ae98b526a None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:37.187168 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103398ca-1982-4d49-a2e1-d28ae98b526a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:37.217336 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-103398ca-1982-4d49-a2e1-d28ae98b526a None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:01:37.219593 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-103398ca-1982-4d49-a2e1-d28ae98b526a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:37.343790 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 946/1883] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:01:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:01:37.351023 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806753ef-1075-48a5-b298-3a1501c454b6 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:37.352641 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-806753ef-1075-48a5-b298-3a1501c454b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:37.461784 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806753ef-1075-48a5-b298-3a1501c454b6 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:37.468806 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2555e66f-c0b9-42eb-a632-c70b4d9de9b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e563fe174db4cc19ae6bf8162cc3d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:37.471530 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2555e66f-c0b9-42eb-a632-c70b4d9de9b3 service neutron] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:37.472372 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2555e66f-c0b9-42eb-a632-c70b4d9de9b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:37.473021 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2555e66f-c0b9-42eb-a632-c70b4d9de9b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:37.493500 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2555e66f-c0b9-42eb-a632-c70b4d9de9b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:37.497092 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2555e66f-c0b9-42eb-a632-c70b4d9de9b3 service neutron] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:37.508870 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 938/1884] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:01:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:01:38.025101 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fae28b-1ae3-4f27-9c70-988cbf02c463 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:38.025657 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fae28b-1ae3-4f27-9c70-988cbf02c463 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:38.026068 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fae28b-1ae3-4f27-9c70-988cbf02c463 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:38.073741 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-35fae28b-1ae3-4f27-9c70-988cbf02c463 None None] MFA Rules not processed for user `ae372b81003e41aeb654da0bcca928d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:01:38.075960 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-35fae28b-1ae3-4f27-9c70-988cbf02c463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:38.109841 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2555e66f-c0b9-42eb-a632-c70b4d9de9b3 service neutron] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:38.111806 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-2555e66f-c0b9-42eb-a632-c70b4d9de9b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:38.149116 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2555e66f-c0b9-42eb-a632-c70b4d9de9b3 service neutron] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:38.155273 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab97076f-e4e9-4227-977d-e60a1fd85788 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:38.155625 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:01:38.155935 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab97076f-e4e9-4227-977d-e60a1fd85788 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:38.156263 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab97076f-e4e9-4227-977d-e60a1fd85788 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:38.156676 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab97076f-e4e9-4227-977d-e60a1fd85788 admin admin] PATH_INFO: `/v3/projects/b4fe478e16b2424c96ad943e4a165e54` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:38.158034 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab97076f-e4e9-4227-977d-e60a1fd85788 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4fe478e16b2424c96ad943e4a165e54)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:38.160301 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab97076f-e4e9-4227-977d-e60a1fd85788 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:38.213165 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 947/1885] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:01:38 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:01:38.236792 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fae28b-1ae3-4f27-9c70-988cbf02c463 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:38.238194 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-ab97076f-e4e9-4227-977d-e60a1fd85788 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4fe478e16b2424c96ad943e4a165e54', 'request_id': 'req-ab97076f-e4e9-4227-977d-e60a1fd85788'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:38.239782 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-35fae28b-1ae3-4f27-9c70-988cbf02c463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:38.252195 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-ab97076f-e4e9-4227-977d-e60a1fd85788 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4fe478e16b2424c96ad943e4a165e54', 'request_id': 'req-ab97076f-e4e9-4227-977d-e60a1fd85788'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:38.271406 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-ab97076f-e4e9-4227-977d-e60a1fd85788 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4fe478e16b2424c96ad943e4a165e54', 'request_id': 'req-ab97076f-e4e9-4227-977d-e60a1fd85788'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:38.291633 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-ab97076f-e4e9-4227-977d-e60a1fd85788 admin admin] The token cache is being invalidate because project b4fe478e16b2424c96ad943e4a165e54 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 12:01:38.292109 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-ab97076f-e4e9-4227-977d-e60a1fd85788 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:38.293966 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-ab97076f-e4e9-4227-977d-e60a1fd85788 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:38.296721 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 939/1886] 10.222.0.59 () {58 vars in 1233 bytes} [Tue Oct 11 12:01:38 2022] DELETE /identity/v3/projects/b4fe478e16b2424c96ad943e4a165e54 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 12:01:38.303978 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2a5faf-35e0-4dee-8bdd-3244cde45f39 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:38.304366 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2a5faf-35e0-4dee-8bdd-3244cde45f39 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:38.304717 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2a5faf-35e0-4dee-8bdd-3244cde45f39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:38.321773 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:01:38.341701 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-de2a5faf-35e0-4dee-8bdd-3244cde45f39 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:01:38.344892 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-de2a5faf-35e0-4dee-8bdd-3244cde45f39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:38.352594 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fae28b-1ae3-4f27-9c70-988cbf02c463 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:38.443904 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 940/1887] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:01:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:01:38.445423 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266c7b7a-64a9-4ffd-95cf-16d785fdeaf5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b1b651c73664d08accc6edf2f79bace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:38.446308 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266c7b7a-64a9-4ffd-95cf-16d785fdeaf5 service cinder] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:38.446806 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266c7b7a-64a9-4ffd-95cf-16d785fdeaf5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:38.447193 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266c7b7a-64a9-4ffd-95cf-16d785fdeaf5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:38.450206 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-266c7b7a-64a9-4ffd-95cf-16d785fdeaf5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:38.455880 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31592bea-442d-4c85-9be4-23ff90f44c52 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:38.456153 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31592bea-442d-4c85-9be4-23ff90f44c52 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:38.456445 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31592bea-442d-4c85-9be4-23ff90f44c52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:38.478482 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-31592bea-442d-4c85-9be4-23ff90f44c52 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:01:38.479658 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-31592bea-442d-4c85-9be4-23ff90f44c52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:38.490368 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 941/1888] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:01:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:01:38.497059 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31592bea-442d-4c85-9be4-23ff90f44c52 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:38.510299 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31592bea-442d-4c85-9be4-23ff90f44c52 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:38.526787 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d14593-34c4-4be6-9de8-19038431ab1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:38.526787 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d14593-34c4-4be6-9de8-19038431ab1f admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:38.526787 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d14593-34c4-4be6-9de8-19038431ab1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:38.526787 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d14593-34c4-4be6-9de8-19038431ab1f admin admin] PATH_INFO: `/v3/domains` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:38.527256 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d14593-34c4-4be6-9de8-19038431ab1f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 12:01:38.527256 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d14593-34c4-4be6-9de8-19038431ab1f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:38.527256 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d14593-34c4-4be6-9de8-19038431ab1f admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:38.536252 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 942/1889] 10.222.0.59 () {58 vars in 1153 bytes} [Tue Oct 11 12:01:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 12:01:38.542581 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d14593-34c4-4be6-9de8-19038431ab1f admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:38.554357 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d14593-34c4-4be6-9de8-19038431ab1f admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:38.560023 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e233508c-1e78-49d2-a502-5a110cd2365d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:38.560654 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e233508c-1e78-49d2-a502-5a110cd2365d admin admin] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:38.560988 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e233508c-1e78-49d2-a502-5a110cd2365d admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:38.561323 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e233508c-1e78-49d2-a502-5a110cd2365d admin admin] PATH_INFO: `/v3/projects` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:38.561946 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e233508c-1e78-49d2-a502-5a110cd2365d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:38.564223 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e233508c-1e78-49d2-a502-5a110cd2365d admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:38.568749 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266c7b7a-64a9-4ffd-95cf-16d785fdeaf5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:38.570653 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266c7b7a-64a9-4ffd-95cf-16d785fdeaf5 service cinder] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:38.590550 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 948/1890] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:01:38 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 11 12:01:38.595598 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 943/1891] 10.222.0.59 () {60 vars in 1153 bytes} [Tue Oct 11 12:01:38 2022] POST /identity/v3/projects => generated 377 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 12:01:38.601830 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266c7b7a-64a9-4ffd-95cf-16d785fdeaf5 service cinder] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:38.626362 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266c7b7a-64a9-4ffd-95cf-16d785fdeaf5 service cinder] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:38.741887 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:01:38.770675 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169794ad-5286-4578-9e44-f044fceaec15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:38.771665 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169794ad-5286-4578-9e44-f044fceaec15 admin admin] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:38.772107 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169794ad-5286-4578-9e44-f044fceaec15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:38.772752 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169794ad-5286-4578-9e44-f044fceaec15 admin admin] PATH_INFO: `/v3/users` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:38.773948 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169794ad-5286-4578-9e44-f044fceaec15 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:38.777958 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169794ad-5286-4578-9e44-f044fceaec15 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:38.821315 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 949/1892] 10.222.0.59 () {60 vars in 1144 bytes} [Tue Oct 11 12:01:38 2022] POST /identity/v3/users => generated 289 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:01:38.823804 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e233508c-1e78-49d2-a502-5a110cd2365d admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:38.836483 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e233508c-1e78-49d2-a502-5a110cd2365d admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:38.843365 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4b6fc0-174d-49d4-93a5-156cecd8c379 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:38.844278 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4b6fc0-174d-49d4-93a5-156cecd8c379 admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:38.844626 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4b6fc0-174d-49d4-93a5-156cecd8c379 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:38.845898 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4b6fc0-174d-49d4-93a5-156cecd8c379 admin admin] PATH_INFO: `/v3/roles` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:38.846938 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4b6fc0-174d-49d4-93a5-156cecd8c379 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:38.849880 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4b6fc0-174d-49d4-93a5-156cecd8c379 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:38.862880 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 944/1893] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:01:38 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:01:38.868864 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169794ad-5286-4578-9e44-f044fceaec15 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:38.884765 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169794ad-5286-4578-9e44-f044fceaec15 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:38.896345 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c428ed49-1478-4d67-ac22-e89ae0c7974a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:38.897330 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c428ed49-1478-4d67-ac22-e89ae0c7974a admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:38.898025 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c428ed49-1478-4d67-ac22-e89ae0c7974a admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:38.898847 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c428ed49-1478-4d67-ac22-e89ae0c7974a admin admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:38.899784 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c428ed49-1478-4d67-ac22-e89ae0c7974a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:38.902693 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c428ed49-1478-4d67-ac22-e89ae0c7974a admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:38.917598 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 950/1894] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:01:38 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:01:38.930656 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4b6fc0-174d-49d4-93a5-156cecd8c379 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:38.947308 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4b6fc0-174d-49d4-93a5-156cecd8c379 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:38.957779 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79646758-c94e-4663-abba-75886e332102 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:38.959028 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79646758-c94e-4663-abba-75886e332102 admin admin] REQUEST_METHOD: `PUT` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:38.959307 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79646758-c94e-4663-abba-75886e332102 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:38.959578 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79646758-c94e-4663-abba-75886e332102 admin admin] PATH_INFO: `/v3/projects/d703dc582b574190a930b8b2c702d705/users/b0479a55cbdf487e977b8a9c51e5bfec/roles/de2d2bd26fff4748a79990f97a107164` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:38.998445 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79646758-c94e-4663-abba-75886e332102 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d703dc582b574190a930b8b2c702d705, user_id=b0479a55cbdf487e977b8a9c51e5bfec, role_id=de2d2bd26fff4748a79990f97a107164)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:39.001416 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79646758-c94e-4663-abba-75886e332102 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:39.031555 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 945/1895] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 12:01:38 2022] PUT /identity/v3/projects/d703dc582b574190a930b8b2c702d705/users/b0479a55cbdf487e977b8a9c51e5bfec/roles/de2d2bd26fff4748a79990f97a107164 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 12:01:39.052601 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0331922c-1f55-4bf0-a33f-8038d44440fb None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:39.059917 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0331922c-1f55-4bf0-a33f-8038d44440fb None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:39.060375 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0331922c-1f55-4bf0-a33f-8038d44440fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:39.123096 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-0331922c-1f55-4bf0-a33f-8038d44440fb None None] MFA Rules not processed for user `b0479a55cbdf487e977b8a9c51e5bfec`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:01:39.124299 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-0331922c-1f55-4bf0-a33f-8038d44440fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:39.231693 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 951/1896] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:01:39 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:01:39.240361 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79646758-c94e-4663-abba-75886e332102 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:39.315972 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79646758-c94e-4663-abba-75886e332102 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:39.321416 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28b3935-bc02-4eda-85ef-cea61e4a4936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:39.322038 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28b3935-bc02-4eda-85ef-cea61e4a4936 admin admin] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:39.322316 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28b3935-bc02-4eda-85ef-cea61e4a4936 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:39.322623 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28b3935-bc02-4eda-85ef-cea61e4a4936 admin admin] PATH_INFO: `/v3/projects` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:39.323270 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28b3935-bc02-4eda-85ef-cea61e4a4936 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:39.325214 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28b3935-bc02-4eda-85ef-cea61e4a4936 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:39.375280 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 946/1897] 10.222.0.59 () {60 vars in 1153 bytes} [Tue Oct 11 12:01:39 2022] POST /identity/v3/projects => generated 375 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:01:39.389414 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0331922c-1f55-4bf0-a33f-8038d44440fb None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:39.403601 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0331922c-1f55-4bf0-a33f-8038d44440fb None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:39.472895 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a966f4-8175-48bf-8746-ebed6ff7ab2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:39.473536 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a966f4-8175-48bf-8746-ebed6ff7ab2d admin admin] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:39.473822 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a966f4-8175-48bf-8746-ebed6ff7ab2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:39.474102 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a966f4-8175-48bf-8746-ebed6ff7ab2d admin admin] PATH_INFO: `/v3/users` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:39.474797 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a966f4-8175-48bf-8746-ebed6ff7ab2d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:39.476847 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a966f4-8175-48bf-8746-ebed6ff7ab2d admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:39.502017 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 952/1898] 10.222.0.59 () {60 vars in 1144 bytes} [Tue Oct 11 12:01:39 2022] POST /identity/v3/users => generated 288 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:01:39.507329 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28b3935-bc02-4eda-85ef-cea61e4a4936 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:39.519470 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28b3935-bc02-4eda-85ef-cea61e4a4936 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:39.527668 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb73c407-1312-495b-8a81-55418f218d4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:39.528647 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb73c407-1312-495b-8a81-55418f218d4f admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:39.529190 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb73c407-1312-495b-8a81-55418f218d4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:39.529664 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb73c407-1312-495b-8a81-55418f218d4f admin admin] PATH_INFO: `/v3/roles` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:39.531005 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb73c407-1312-495b-8a81-55418f218d4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:39.534029 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb73c407-1312-495b-8a81-55418f218d4f admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:39.556387 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 947/1899] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:01:39 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 11 12:01:39.560666 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a966f4-8175-48bf-8746-ebed6ff7ab2d admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:39.573743 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a966f4-8175-48bf-8746-ebed6ff7ab2d admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:39.581805 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed555da4-3915-4c94-af16-de2cdbfe48cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:39.583353 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed555da4-3915-4c94-af16-de2cdbfe48cd admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:39.583963 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed555da4-3915-4c94-af16-de2cdbfe48cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:39.584552 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed555da4-3915-4c94-af16-de2cdbfe48cd admin admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:39.585957 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed555da4-3915-4c94-af16-de2cdbfe48cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:39.588053 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed555da4-3915-4c94-af16-de2cdbfe48cd admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:39.608108 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 953/1900] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:01:39 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:01:39.612847 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb73c407-1312-495b-8a81-55418f218d4f admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:39.618313 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:01:39.627815 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb73c407-1312-495b-8a81-55418f218d4f admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:39.635837 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4c434f-40e7-4a1d-bdf1-df19662d16de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:39.636676 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4c434f-40e7-4a1d-bdf1-df19662d16de admin admin] REQUEST_METHOD: `PUT` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:39.637092 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4c434f-40e7-4a1d-bdf1-df19662d16de admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:39.637460 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4c434f-40e7-4a1d-bdf1-df19662d16de admin admin] PATH_INFO: `/v3/projects/edc4e6ba289f4cc2865930d616c0fd20/users/2fd2508bd91346d29af1bed2ff856448/roles/de2d2bd26fff4748a79990f97a107164` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:39.661558 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4c434f-40e7-4a1d-bdf1-df19662d16de admin admin] RBAC: Authorizing `identity:create_grant(project_id=edc4e6ba289f4cc2865930d616c0fd20, user_id=2fd2508bd91346d29af1bed2ff856448, role_id=de2d2bd26fff4748a79990f97a107164)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:39.664563 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4c434f-40e7-4a1d-bdf1-df19662d16de admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:39.680374 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 948/1901] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 12:01:39 2022] PUT /identity/v3/projects/edc4e6ba289f4cc2865930d616c0fd20/users/2fd2508bd91346d29af1bed2ff856448/roles/de2d2bd26fff4748a79990f97a107164 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:01:39.691460 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfff7cb-6fbb-4403-bfba-7cf2eb19351d None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:39.691888 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfff7cb-6fbb-4403-bfba-7cf2eb19351d None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:39.692223 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfff7cb-6fbb-4403-bfba-7cf2eb19351d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:39.727319 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-0bfff7cb-6fbb-4403-bfba-7cf2eb19351d None None] MFA Rules not processed for user `2fd2508bd91346d29af1bed2ff856448`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:01:39.729207 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-0bfff7cb-6fbb-4403-bfba-7cf2eb19351d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:39.757704 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:01:39.815112 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 954/1902] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:01:39 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:01:39.819995 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4c434f-40e7-4a1d-bdf1-df19662d16de admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:39.888155 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4c434f-40e7-4a1d-bdf1-df19662d16de admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:39.893093 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66395a5-1cdf-4bcf-bdf4-590f1c541357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:39.893718 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66395a5-1cdf-4bcf-bdf4-590f1c541357 admin admin] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:39.893993 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66395a5-1cdf-4bcf-bdf4-590f1c541357 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:39.894267 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66395a5-1cdf-4bcf-bdf4-590f1c541357 admin admin] PATH_INFO: `/v3/projects` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:39.894942 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66395a5-1cdf-4bcf-bdf4-590f1c541357 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:39.896868 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66395a5-1cdf-4bcf-bdf4-590f1c541357 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:39.929695 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 949/1903] 10.222.0.59 () {60 vars in 1153 bytes} [Tue Oct 11 12:01:39 2022] POST /identity/v3/projects => generated 373 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:01:39.934707 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfff7cb-6fbb-4403-bfba-7cf2eb19351d None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:39.946839 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfff7cb-6fbb-4403-bfba-7cf2eb19351d None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:40.010402 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5f179f-d000-43f7-a23f-140a88bebfd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:40.011162 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5f179f-d000-43f7-a23f-140a88bebfd3 admin admin] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:40.011525 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5f179f-d000-43f7-a23f-140a88bebfd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:40.011973 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5f179f-d000-43f7-a23f-140a88bebfd3 admin admin] PATH_INFO: `/v3/users` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:40.012929 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5f179f-d000-43f7-a23f-140a88bebfd3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:40.014998 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5f179f-d000-43f7-a23f-140a88bebfd3 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:40.043119 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 955/1904] 10.222.0.59 () {60 vars in 1144 bytes} [Tue Oct 11 12:01:39 2022] POST /identity/v3/users => generated 293 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:01:40.047226 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66395a5-1cdf-4bcf-bdf4-590f1c541357 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:40.058254 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66395a5-1cdf-4bcf-bdf4-590f1c541357 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:40.063181 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db3d1d3-f2bd-465d-ac79-308d86ec5d56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:40.063802 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db3d1d3-f2bd-465d-ac79-308d86ec5d56 admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:40.064076 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db3d1d3-f2bd-465d-ac79-308d86ec5d56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:40.064344 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db3d1d3-f2bd-465d-ac79-308d86ec5d56 admin admin] PATH_INFO: `/v3/roles` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:40.064993 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db3d1d3-f2bd-465d-ac79-308d86ec5d56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:40.066663 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db3d1d3-f2bd-465d-ac79-308d86ec5d56 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:40.078966 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 950/1905] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:01:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:01:40.083248 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5f179f-d000-43f7-a23f-140a88bebfd3 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:40.097303 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5f179f-d000-43f7-a23f-140a88bebfd3 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:40.103215 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3af286-109d-4c02-a25b-8f0e93b4a29f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:40.104315 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3af286-109d-4c02-a25b-8f0e93b4a29f admin admin] REQUEST_METHOD: `PUT` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:40.104680 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3af286-109d-4c02-a25b-8f0e93b4a29f admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:40.105755 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3af286-109d-4c02-a25b-8f0e93b4a29f admin admin] PATH_INFO: `/v3/projects/97a53cd24ef847b6952d8344fb87168b/users/d48cd86c413743afb8713a3789d987ed/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:40.126245 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3af286-109d-4c02-a25b-8f0e93b4a29f admin admin] RBAC: Authorizing `identity:create_grant(project_id=97a53cd24ef847b6952d8344fb87168b, user_id=d48cd86c413743afb8713a3789d987ed, role_id=5cf1f4bd2fef46ab9c6eff56d52f9f69)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:40.130430 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3af286-109d-4c02-a25b-8f0e93b4a29f admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:40.146968 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 956/1906] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 12:01:40 2022] PUT /identity/v3/projects/97a53cd24ef847b6952d8344fb87168b/users/d48cd86c413743afb8713a3789d987ed/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:01:40.151009 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db3d1d3-f2bd-465d-ac79-308d86ec5d56 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:40.162298 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db3d1d3-f2bd-465d-ac79-308d86ec5d56 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:40.270079 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40520b22-cda3-4d7d-aa29-e3cfa41b4819 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:40.270377 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40520b22-cda3-4d7d-aa29-e3cfa41b4819 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:40.270688 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40520b22-cda3-4d7d-aa29-e3cfa41b4819 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:40.290027 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b49e2ee-fd96-4d1f-b27a-23c2ef29bc65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:40.290867 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b49e2ee-fd96-4d1f-b27a-23c2ef29bc65 admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:40.291221 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b49e2ee-fd96-4d1f-b27a-23c2ef29bc65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:40.292217 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b49e2ee-fd96-4d1f-b27a-23c2ef29bc65 admin admin] PATH_INFO: `/v3/roles` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:40.293058 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b49e2ee-fd96-4d1f-b27a-23c2ef29bc65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:40.294941 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b49e2ee-fd96-4d1f-b27a-23c2ef29bc65 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:40.300114 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-40520b22-cda3-4d7d-aa29-e3cfa41b4819 None None] MFA Rules not processed for user `ae372b81003e41aeb654da0bcca928d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:01:40.301659 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-40520b22-cda3-4d7d-aa29-e3cfa41b4819 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:40.310908 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 951/1907] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:01:40 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:01:40.317026 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b49e2ee-fd96-4d1f-b27a-23c2ef29bc65 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:40.329164 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b49e2ee-fd96-4d1f-b27a-23c2ef29bc65 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:40.336432 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de99f87d-199c-4be2-8617-5bd0f0183a1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:40.337740 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de99f87d-199c-4be2-8617-5bd0f0183a1f admin admin] REQUEST_METHOD: `PUT` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:40.338123 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de99f87d-199c-4be2-8617-5bd0f0183a1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:40.338534 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de99f87d-199c-4be2-8617-5bd0f0183a1f admin admin] PATH_INFO: `/v3/projects/97a53cd24ef847b6952d8344fb87168b/users/d48cd86c413743afb8713a3789d987ed/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:40.344855 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de99f87d-199c-4be2-8617-5bd0f0183a1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=97a53cd24ef847b6952d8344fb87168b, user_id=d48cd86c413743afb8713a3789d987ed, role_id=5cf1f4bd2fef46ab9c6eff56d52f9f69)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:40.348277 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de99f87d-199c-4be2-8617-5bd0f0183a1f admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:40.371788 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 952/1908] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 12:01:40 2022] PUT /identity/v3/projects/97a53cd24ef847b6952d8344fb87168b/users/d48cd86c413743afb8713a3789d987ed/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 12:01:40.382825 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daf8aff-67e1-43d3-8390-0d834da30786 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:40.383198 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daf8aff-67e1-43d3-8390-0d834da30786 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:40.383611 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daf8aff-67e1-43d3-8390-0d834da30786 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:40.402866 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 957/1909] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:01:40 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 12:01:40.443283 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-5daf8aff-67e1-43d3-8390-0d834da30786 None None] MFA Rules not processed for user `d48cd86c413743afb8713a3789d987ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:01:40.466660 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40520b22-cda3-4d7d-aa29-e3cfa41b4819 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:40.468209 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-5daf8aff-67e1-43d3-8390-0d834da30786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:40.474600 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-40520b22-cda3-4d7d-aa29-e3cfa41b4819 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:40.500526 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40520b22-cda3-4d7d-aa29-e3cfa41b4819 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:40.515159 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:01:40.565951 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:01:40.604797 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 953/1910] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:01:40 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 12:01:40.610974 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64691ba6-f13e-405f-bc81-526524da8204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:40.611949 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64691ba6-f13e-405f-bc81-526524da8204 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:40.612272 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64691ba6-f13e-405f-bc81-526524da8204 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:40.612636 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64691ba6-f13e-405f-bc81-526524da8204 admin admin] PATH_INFO: `/v3/users/1b664bef72e64f508d2cc56fd6a210fd` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:40.614207 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64691ba6-f13e-405f-bc81-526524da8204 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b664bef72e64f508d2cc56fd6a210fd)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:40.616162 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64691ba6-f13e-405f-bc81-526524da8204 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:40.630582 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daf8aff-67e1-43d3-8390-0d834da30786 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:40.633698 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-5daf8aff-67e1-43d3-8390-0d834da30786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:40.693924 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-64691ba6-f13e-405f-bc81-526524da8204 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b664bef72e64f508d2cc56fd6a210fd'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:40.722414 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-64691ba6-f13e-405f-bc81-526524da8204 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b664bef72e64f508d2cc56fd6a210fd'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:40.725882 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-64691ba6-f13e-405f-bc81-526524da8204 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b664bef72e64f508d2cc56fd6a210fd'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:40.743910 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-64691ba6-f13e-405f-bc81-526524da8204 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b664bef72e64f508d2cc56fd6a210fd'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:40.745527 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-64691ba6-f13e-405f-bc81-526524da8204 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b664bef72e64f508d2cc56fd6a210fd'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:40.777000 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daf8aff-67e1-43d3-8390-0d834da30786 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:40.778630 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 958/1911] 10.222.0.59 () {58 vars in 1224 bytes} [Tue Oct 11 12:01:40 2022] DELETE /identity/v3/users/1b664bef72e64f508d2cc56fd6a210fd => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 12:01:40.784923 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ab9131-2963-49b0-8295-260131369caf None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:40.785721 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ab9131-2963-49b0-8295-260131369caf None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:40.786201 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ab9131-2963-49b0-8295-260131369caf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:40.827206 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-f1ab9131-2963-49b0-8295-260131369caf None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:01:40.829673 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-f1ab9131-2963-49b0-8295-260131369caf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:40.855339 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce2c516-d3b8-4d5e-b7df-236092049223 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b1b651c73664d08accc6edf2f79bace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:40.856516 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce2c516-d3b8-4d5e-b7df-236092049223 service cinder] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:40.856516 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce2c516-d3b8-4d5e-b7df-236092049223 service cinder] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:40.856703 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce2c516-d3b8-4d5e-b7df-236092049223 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:40.859079 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-cce2c516-d3b8-4d5e-b7df-236092049223 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:40.916397 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 959/1912] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:01:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:01:40.927816 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ab9131-2963-49b0-8295-260131369caf None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:40.932463 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-f1ab9131-2963-49b0-8295-260131369caf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:40.966599 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce2c516-d3b8-4d5e-b7df-236092049223 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:40.966599 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce2c516-d3b8-4d5e-b7df-236092049223 service cinder] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:40.984307 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 954/1913] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:01:40 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:01:41.028913 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ab9131-2963-49b0-8295-260131369caf None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:41.033325 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc01d9e4-b452-457a-a5e3-a60986021516 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e563fe174db4cc19ae6bf8162cc3d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:41.033963 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc01d9e4-b452-457a-a5e3-a60986021516 service neutron] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:41.034253 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc01d9e4-b452-457a-a5e3-a60986021516 service neutron] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:41.034639 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc01d9e4-b452-457a-a5e3-a60986021516 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:41.047851 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc01d9e4-b452-457a-a5e3-a60986021516 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:41.052151 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc01d9e4-b452-457a-a5e3-a60986021516 service neutron] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:41.060092 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:01:41.070876 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 960/1914] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:01:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:01:41.639251 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce2c516-d3b8-4d5e-b7df-236092049223 service cinder] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:41.643171 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-cce2c516-d3b8-4d5e-b7df-236092049223 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:41.677155 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce2c516-d3b8-4d5e-b7df-236092049223 service cinder] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:41.685621 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c17752-63e8-4c1c-96e0-810a03bd12e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:41.686452 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c17752-63e8-4c1c-96e0-810a03bd12e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:41.686828 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c17752-63e8-4c1c-96e0-810a03bd12e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:41.687171 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c17752-63e8-4c1c-96e0-810a03bd12e2 admin admin] PATH_INFO: `/v3/projects/6d67d978bce04debbd1ce49e589a735a` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:41.688585 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c17752-63e8-4c1c-96e0-810a03bd12e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d67d978bce04debbd1ce49e589a735a)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:41.690641 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c17752-63e8-4c1c-96e0-810a03bd12e2 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:41.737017 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:01:41.768803 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-a9c17752-63e8-4c1c-96e0-810a03bd12e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d67d978bce04debbd1ce49e589a735a', 'request_id': 'req-a9c17752-63e8-4c1c-96e0-810a03bd12e2'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:41.782106 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-a9c17752-63e8-4c1c-96e0-810a03bd12e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d67d978bce04debbd1ce49e589a735a', 'request_id': 'req-a9c17752-63e8-4c1c-96e0-810a03bd12e2'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:41.794888 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-a9c17752-63e8-4c1c-96e0-810a03bd12e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d67d978bce04debbd1ce49e589a735a', 'request_id': 'req-a9c17752-63e8-4c1c-96e0-810a03bd12e2'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:41.816848 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-a9c17752-63e8-4c1c-96e0-810a03bd12e2 admin admin] The token cache is being invalidate because project 6d67d978bce04debbd1ce49e589a735a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 12:01:41.817183 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-a9c17752-63e8-4c1c-96e0-810a03bd12e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:41.818265 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-a9c17752-63e8-4c1c-96e0-810a03bd12e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:41.820252 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 955/1915] 10.222.0.59 () {58 vars in 1233 bytes} [Tue Oct 11 12:01:41 2022] DELETE /identity/v3/projects/6d67d978bce04debbd1ce49e589a735a => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 12:01:41.825978 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa03780-0212-49f1-a49c-bff8a509d1cc None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:41.826516 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa03780-0212-49f1-a49c-bff8a509d1cc None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:41.826630 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa03780-0212-49f1-a49c-bff8a509d1cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:41.850398 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-8fa03780-0212-49f1-a49c-bff8a509d1cc None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:01:41.851700 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-8fa03780-0212-49f1-a49c-bff8a509d1cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:41.922650 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 961/1916] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:01:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:01:41.931446 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9122956-5582-4a74-b166-da7ab4a75e5c None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:41.931685 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9122956-5582-4a74-b166-da7ab4a75e5c None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:41.931946 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9122956-5582-4a74-b166-da7ab4a75e5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:41.954741 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-b9122956-5582-4a74-b166-da7ab4a75e5c None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:01:41.956034 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-b9122956-5582-4a74-b166-da7ab4a75e5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:41.965814 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 956/1917] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:01:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:01:41.972991 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa03780-0212-49f1-a49c-bff8a509d1cc None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:41.990583 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa03780-0212-49f1-a49c-bff8a509d1cc None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:41.997645 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248705b5-7c9d-4e4a-866a-6ec007835570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:41.998272 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248705b5-7c9d-4e4a-866a-6ec007835570 admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:41.998578 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248705b5-7c9d-4e4a-866a-6ec007835570 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:41.999002 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248705b5-7c9d-4e4a-866a-6ec007835570 admin admin] PATH_INFO: `/v3/domains` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:41.999672 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248705b5-7c9d-4e4a-866a-6ec007835570 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 12:01:41.999980 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248705b5-7c9d-4e4a-866a-6ec007835570 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:42.001770 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248705b5-7c9d-4e4a-866a-6ec007835570 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:42.018270 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 962/1918] 10.222.0.59 () {58 vars in 1153 bytes} [Tue Oct 11 12:01:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 12:01:42.019558 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9122956-5582-4a74-b166-da7ab4a75e5c None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:42.033925 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9122956-5582-4a74-b166-da7ab4a75e5c None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:42.038980 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628c3a92-a534-4b79-b21c-fb37cb1f77ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:42.039679 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628c3a92-a534-4b79-b21c-fb37cb1f77ef admin admin] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:42.039868 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628c3a92-a534-4b79-b21c-fb37cb1f77ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:42.040142 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628c3a92-a534-4b79-b21c-fb37cb1f77ef admin admin] PATH_INFO: `/v3/projects` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:42.040785 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628c3a92-a534-4b79-b21c-fb37cb1f77ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:42.042752 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628c3a92-a534-4b79-b21c-fb37cb1f77ef admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:42.077968 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 957/1919] 10.222.0.59 () {60 vars in 1153 bytes} [Tue Oct 11 12:01:42 2022] POST /identity/v3/projects => generated 369 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:01:42.079232 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248705b5-7c9d-4e4a-866a-6ec007835570 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:42.089740 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248705b5-7c9d-4e4a-866a-6ec007835570 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:42.160288 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b617a0-9cce-4890-9bbc-cabc45cd1c97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:42.160923 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b617a0-9cce-4890-9bbc-cabc45cd1c97 admin admin] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:42.161256 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b617a0-9cce-4890-9bbc-cabc45cd1c97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:42.161483 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b617a0-9cce-4890-9bbc-cabc45cd1c97 admin admin] PATH_INFO: `/v3/users` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:42.162161 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b617a0-9cce-4890-9bbc-cabc45cd1c97 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:42.164228 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b617a0-9cce-4890-9bbc-cabc45cd1c97 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:42.189048 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 963/1920] 10.222.0.59 () {60 vars in 1144 bytes} [Tue Oct 11 12:01:42 2022] POST /identity/v3/users => generated 285 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 12:01:42.194031 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628c3a92-a534-4b79-b21c-fb37cb1f77ef admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:42.207844 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628c3a92-a534-4b79-b21c-fb37cb1f77ef admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:42.214227 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0adc7b-0bad-4f35-af1f-b9f2d9aaf75b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:42.215227 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0adc7b-0bad-4f35-af1f-b9f2d9aaf75b admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:42.215639 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0adc7b-0bad-4f35-af1f-b9f2d9aaf75b admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:42.215992 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0adc7b-0bad-4f35-af1f-b9f2d9aaf75b admin admin] PATH_INFO: `/v3/roles` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:42.216691 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0adc7b-0bad-4f35-af1f-b9f2d9aaf75b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:42.218669 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0adc7b-0bad-4f35-af1f-b9f2d9aaf75b admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:42.238185 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 958/1921] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:01:42 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:01:42.242384 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b617a0-9cce-4890-9bbc-cabc45cd1c97 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:42.254942 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b617a0-9cce-4890-9bbc-cabc45cd1c97 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:42.262341 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33de79b6-9ca2-4617-b25b-42026c3722d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:42.263140 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33de79b6-9ca2-4617-b25b-42026c3722d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:42.263423 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33de79b6-9ca2-4617-b25b-42026c3722d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:42.263698 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33de79b6-9ca2-4617-b25b-42026c3722d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:42.264362 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33de79b6-9ca2-4617-b25b-42026c3722d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:42.266176 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33de79b6-9ca2-4617-b25b-42026c3722d9 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:42.280691 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 964/1922] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:01:42 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:01:42.286088 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0adc7b-0bad-4f35-af1f-b9f2d9aaf75b admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:42.303348 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0adc7b-0bad-4f35-af1f-b9f2d9aaf75b admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:42.311408 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a682bc7-5198-4b55-9642-41de7faab688 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:42.312543 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a682bc7-5198-4b55-9642-41de7faab688 admin admin] REQUEST_METHOD: `PUT` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:42.313069 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a682bc7-5198-4b55-9642-41de7faab688 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:42.313440 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a682bc7-5198-4b55-9642-41de7faab688 admin admin] PATH_INFO: `/v3/projects/deb7156d7ce349799b9040cad51098e1/users/42c13366d0644388a703beaab38a6ead/roles/de2d2bd26fff4748a79990f97a107164` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:42.342766 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a682bc7-5198-4b55-9642-41de7faab688 admin admin] RBAC: Authorizing `identity:create_grant(project_id=deb7156d7ce349799b9040cad51098e1, user_id=42c13366d0644388a703beaab38a6ead, role_id=de2d2bd26fff4748a79990f97a107164)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:42.345879 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a682bc7-5198-4b55-9642-41de7faab688 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:42.369908 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 959/1923] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 12:01:42 2022] PUT /identity/v3/projects/deb7156d7ce349799b9040cad51098e1/users/42c13366d0644388a703beaab38a6ead/roles/de2d2bd26fff4748a79990f97a107164 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:01:42.388985 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e697f51e-4e8c-4d2b-9cd0-b19fa6333892 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:42.389470 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e697f51e-4e8c-4d2b-9cd0-b19fa6333892 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:42.389970 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e697f51e-4e8c-4d2b-9cd0-b19fa6333892 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:42.431202 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-e697f51e-4e8c-4d2b-9cd0-b19fa6333892 None None] MFA Rules not processed for user `42c13366d0644388a703beaab38a6ead`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:01:42.432432 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-e697f51e-4e8c-4d2b-9cd0-b19fa6333892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:42.496854 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:01:42.509596 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 965/1924] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:01:42 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:01:42.527921 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a682bc7-5198-4b55-9642-41de7faab688 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:42.530887 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-3a682bc7-5198-4b55-9642-41de7faab688 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:42.661863 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a682bc7-5198-4b55-9642-41de7faab688 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:42.668126 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b943f5-cfce-46ff-91b1-92d9291ed777 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91a00658b79a42bd91a22a2508b5de34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:42.668894 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b943f5-cfce-46ff-91b1-92d9291ed777 service glance] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:42.669184 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b943f5-cfce-46ff-91b1-92d9291ed777 service glance] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:42.669457 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b943f5-cfce-46ff-91b1-92d9291ed777 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:42.679530 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b943f5-cfce-46ff-91b1-92d9291ed777 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:42.680963 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b943f5-cfce-46ff-91b1-92d9291ed777 service glance] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:42.696218 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 960/1925] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:01:42 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:01:46.108348 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e697f51e-4e8c-4d2b-9cd0-b19fa6333892 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:46.114599 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-e697f51e-4e8c-4d2b-9cd0-b19fa6333892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:46.268507 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e697f51e-4e8c-4d2b-9cd0-b19fa6333892 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:46.277915 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed6ba77-6055-4756-9bb7-6cf485da39cf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b1b651c73664d08accc6edf2f79bace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:46.280220 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed6ba77-6055-4756-9bb7-6cf485da39cf service cinder] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:46.280302 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed6ba77-6055-4756-9bb7-6cf485da39cf service cinder] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:46.280806 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed6ba77-6055-4756-9bb7-6cf485da39cf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:46.284097 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-eed6ba77-6055-4756-9bb7-6cf485da39cf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:46.370504 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed6ba77-6055-4756-9bb7-6cf485da39cf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:46.372608 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed6ba77-6055-4756-9bb7-6cf485da39cf service cinder] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:46.394782 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 966/1926] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:01:46 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:01:47.691275 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b943f5-cfce-46ff-91b1-92d9291ed777 service glance] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:47.718398 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b943f5-cfce-46ff-91b1-92d9291ed777 service glance] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:47.723904 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a7925f-c49b-4740-a655-5c60099e3e21 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b1b651c73664d08accc6edf2f79bace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:47.724762 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a7925f-c49b-4740-a655-5c60099e3e21 service cinder] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:47.725194 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a7925f-c49b-4740-a655-5c60099e3e21 service cinder] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:47.725638 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a7925f-c49b-4740-a655-5c60099e3e21 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:47.729145 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-45a7925f-c49b-4740-a655-5c60099e3e21 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:47.796256 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:01:47.817047 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed6ba77-6055-4756-9bb7-6cf485da39cf service cinder] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:47.819450 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-eed6ba77-6055-4756-9bb7-6cf485da39cf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:47.860656 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a7925f-c49b-4740-a655-5c60099e3e21 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:47.863701 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a7925f-c49b-4740-a655-5c60099e3e21 service cinder] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:47.884347 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 961/1927] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:01:47 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:01:47.929100 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed6ba77-6055-4756-9bb7-6cf485da39cf service cinder] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:47.937544 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce22f31c-03dd-4ef4-96b4-89d9fa0a1d70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:47.938590 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce22f31c-03dd-4ef4-96b4-89d9fa0a1d70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:47.939144 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce22f31c-03dd-4ef4-96b4-89d9fa0a1d70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:47.939585 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce22f31c-03dd-4ef4-96b4-89d9fa0a1d70 admin admin] PATH_INFO: `/v3/users/3f91177e52734ab79932d8ca6603b7c4` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:47.941144 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce22f31c-03dd-4ef4-96b4-89d9fa0a1d70 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f91177e52734ab79932d8ca6603b7c4)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:47.957172 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce22f31c-03dd-4ef4-96b4-89d9fa0a1d70 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:48.122753 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-ce22f31c-03dd-4ef4-96b4-89d9fa0a1d70 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f91177e52734ab79932d8ca6603b7c4'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:48.134086 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:01:48.192608 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-ce22f31c-03dd-4ef4-96b4-89d9fa0a1d70 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f91177e52734ab79932d8ca6603b7c4'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:48.194181 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-ce22f31c-03dd-4ef4-96b4-89d9fa0a1d70 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f91177e52734ab79932d8ca6603b7c4'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:48.218341 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-ce22f31c-03dd-4ef4-96b4-89d9fa0a1d70 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f91177e52734ab79932d8ca6603b7c4'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:48.220244 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-ce22f31c-03dd-4ef4-96b4-89d9fa0a1d70 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f91177e52734ab79932d8ca6603b7c4'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:48.251146 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 967/1928] 10.222.0.59 () {58 vars in 1224 bytes} [Tue Oct 11 12:01:47 2022] DELETE /identity/v3/users/3f91177e52734ab79932d8ca6603b7c4 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 12:01:48.258130 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9195bd-8224-4c7c-8bdd-d6db64b0e088 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:48.258984 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9195bd-8224-4c7c-8bdd-d6db64b0e088 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:48.259704 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9195bd-8224-4c7c-8bdd-d6db64b0e088 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:48.293725 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-dc9195bd-8224-4c7c-8bdd-d6db64b0e088 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:01:48.295200 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-dc9195bd-8224-4c7c-8bdd-d6db64b0e088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:48.377127 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 962/1929] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:01:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:01:48.428225 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce22f31c-03dd-4ef4-96b4-89d9fa0a1d70 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:48.433240 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-ce22f31c-03dd-4ef4-96b4-89d9fa0a1d70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:48.613151 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce22f31c-03dd-4ef4-96b4-89d9fa0a1d70 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:48.617866 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc5ec58-e870-46e8-8b2c-fe51914bbeba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e563fe174db4cc19ae6bf8162cc3d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:48.618430 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc5ec58-e870-46e8-8b2c-fe51914bbeba service neutron] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:48.618826 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc5ec58-e870-46e8-8b2c-fe51914bbeba service neutron] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:48.619113 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc5ec58-e870-46e8-8b2c-fe51914bbeba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:48.631500 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc5ec58-e870-46e8-8b2c-fe51914bbeba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:48.633175 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc5ec58-e870-46e8-8b2c-fe51914bbeba service neutron] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:48.658664 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 968/1930] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:01:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:01:49.026030 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9195bd-8224-4c7c-8bdd-d6db64b0e088 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:49.027866 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-dc9195bd-8224-4c7c-8bdd-d6db64b0e088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:49.049958 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9195bd-8224-4c7c-8bdd-d6db64b0e088 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:49.057485 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4363aac-e22a-4783-843e-baf3682a556b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:49.058465 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4363aac-e22a-4783-843e-baf3682a556b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:49.058898 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4363aac-e22a-4783-843e-baf3682a556b admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:49.059161 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4363aac-e22a-4783-843e-baf3682a556b admin admin] PATH_INFO: `/v3/projects/0b1ac274b4bb4997a54db5eebd03627d` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:49.061094 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4363aac-e22a-4783-843e-baf3682a556b admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b1ac274b4bb4997a54db5eebd03627d)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:49.064758 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4363aac-e22a-4783-843e-baf3682a556b admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:49.126669 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-d4363aac-e22a-4783-843e-baf3682a556b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b1ac274b4bb4997a54db5eebd03627d', 'request_id': 'req-d4363aac-e22a-4783-843e-baf3682a556b'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:49.132689 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-d4363aac-e22a-4783-843e-baf3682a556b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b1ac274b4bb4997a54db5eebd03627d', 'request_id': 'req-d4363aac-e22a-4783-843e-baf3682a556b'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:49.142598 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-d4363aac-e22a-4783-843e-baf3682a556b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b1ac274b4bb4997a54db5eebd03627d', 'request_id': 'req-d4363aac-e22a-4783-843e-baf3682a556b'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:49.160206 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-d4363aac-e22a-4783-843e-baf3682a556b admin admin] The token cache is being invalidate because project 0b1ac274b4bb4997a54db5eebd03627d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 12:01:49.160543 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-d4363aac-e22a-4783-843e-baf3682a556b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:49.161384 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-d4363aac-e22a-4783-843e-baf3682a556b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:49.163071 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 963/1931] 10.222.0.59 () {58 vars in 1233 bytes} [Tue Oct 11 12:01:49 2022] DELETE /identity/v3/projects/0b1ac274b4bb4997a54db5eebd03627d => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:01:49.168625 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc5ec58-e870-46e8-8b2c-fe51914bbeba service neutron] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:49.181063 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc5ec58-e870-46e8-8b2c-fe51914bbeba service neutron] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:49.272682 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce8e5f8-4e8e-4fc3-a3ad-f57663906d64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:49.273701 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce8e5f8-4e8e-4fc3-a3ad-f57663906d64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:49.274156 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce8e5f8-4e8e-4fc3-a3ad-f57663906d64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:49.274652 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce8e5f8-4e8e-4fc3-a3ad-f57663906d64 admin admin] PATH_INFO: `/v3/users/e33c0fdbf5aa455dafc95427e8a9544d` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:49.276858 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce8e5f8-4e8e-4fc3-a3ad-f57663906d64 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e33c0fdbf5aa455dafc95427e8a9544d)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:49.281023 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce8e5f8-4e8e-4fc3-a3ad-f57663906d64 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:49.361291 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-1ce8e5f8-4e8e-4fc3-a3ad-f57663906d64 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e33c0fdbf5aa455dafc95427e8a9544d'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:49.382079 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-1ce8e5f8-4e8e-4fc3-a3ad-f57663906d64 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e33c0fdbf5aa455dafc95427e8a9544d'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:49.383160 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-1ce8e5f8-4e8e-4fc3-a3ad-f57663906d64 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e33c0fdbf5aa455dafc95427e8a9544d'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:49.394847 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-1ce8e5f8-4e8e-4fc3-a3ad-f57663906d64 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e33c0fdbf5aa455dafc95427e8a9544d'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:49.395725 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-1ce8e5f8-4e8e-4fc3-a3ad-f57663906d64 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e33c0fdbf5aa455dafc95427e8a9544d'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:49.397391 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:01:49.424900 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 969/1932] 10.222.0.59 () {58 vars in 1224 bytes} [Tue Oct 11 12:01:49 2022] DELETE /identity/v3/users/e33c0fdbf5aa455dafc95427e8a9544d => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:01:49.652749 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4363aac-e22a-4783-843e-baf3682a556b admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:49.658711 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-d4363aac-e22a-4783-843e-baf3682a556b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:49.728684 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4363aac-e22a-4783-843e-baf3682a556b admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:49.857937 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:01:49.865904 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07129f48-37a8-44e9-b104-b23ecffc1418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:49.867299 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07129f48-37a8-44e9-b104-b23ecffc1418 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:49.867779 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07129f48-37a8-44e9-b104-b23ecffc1418 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:49.868389 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07129f48-37a8-44e9-b104-b23ecffc1418 admin admin] PATH_INFO: `/v3/users/05738791f4a34b808b2e02f491a7dc76` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:49.870282 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07129f48-37a8-44e9-b104-b23ecffc1418 admin admin] RBAC: Authorizing `identity:delete_user(user_id=05738791f4a34b808b2e02f491a7dc76)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:49.873052 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07129f48-37a8-44e9-b104-b23ecffc1418 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:49.933128 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce8e5f8-4e8e-4fc3-a3ad-f57663906d64 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:49.935238 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-1ce8e5f8-4e8e-4fc3-a3ad-f57663906d64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:49.955335 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce8e5f8-4e8e-4fc3-a3ad-f57663906d64 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:49.977976 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3387de-590b-4539-a302-ef32dd099767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:49.977976 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3387de-590b-4539-a302-ef32dd099767 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:49.978295 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3387de-590b-4539-a302-ef32dd099767 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:49.979278 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3387de-590b-4539-a302-ef32dd099767 admin admin] PATH_INFO: `/v3/projects/bc8eab398ee643eebae35cfb79ca4f7e` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:49.980505 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3387de-590b-4539-a302-ef32dd099767 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc8eab398ee643eebae35cfb79ca4f7e)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:49.982974 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3387de-590b-4539-a302-ef32dd099767 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:50.008030 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-07129f48-37a8-44e9-b104-b23ecffc1418 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05738791f4a34b808b2e02f491a7dc76'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:50.054164 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-07129f48-37a8-44e9-b104-b23ecffc1418 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05738791f4a34b808b2e02f491a7dc76'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:50.068872 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-07129f48-37a8-44e9-b104-b23ecffc1418 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05738791f4a34b808b2e02f491a7dc76'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:50.069776 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-07129f48-37a8-44e9-b104-b23ecffc1418 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05738791f4a34b808b2e02f491a7dc76'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:50.084600 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-07129f48-37a8-44e9-b104-b23ecffc1418 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05738791f4a34b808b2e02f491a7dc76'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:50.104378 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 964/1933] 10.222.0.59 () {58 vars in 1224 bytes} [Tue Oct 11 12:01:49 2022] DELETE /identity/v3/users/05738791f4a34b808b2e02f491a7dc76 => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:01:50.106963 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-1b3387de-590b-4539-a302-ef32dd099767 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc8eab398ee643eebae35cfb79ca4f7e', 'request_id': 'req-1b3387de-590b-4539-a302-ef32dd099767'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:50.107140 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b23c63b-76f7-4150-96ce-9686235c1093 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:50.107140 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b23c63b-76f7-4150-96ce-9686235c1093 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:50.107140 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b23c63b-76f7-4150-96ce-9686235c1093 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:50.121629 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-1b3387de-590b-4539-a302-ef32dd099767 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc8eab398ee643eebae35cfb79ca4f7e', 'request_id': 'req-1b3387de-590b-4539-a302-ef32dd099767'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:50.130683 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-1b3387de-590b-4539-a302-ef32dd099767 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc8eab398ee643eebae35cfb79ca4f7e', 'request_id': 'req-1b3387de-590b-4539-a302-ef32dd099767'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:50.138914 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-3b23c63b-76f7-4150-96ce-9686235c1093 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:01:50.140202 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-3b23c63b-76f7-4150-96ce-9686235c1093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:50.151740 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-1b3387de-590b-4539-a302-ef32dd099767 admin admin] The token cache is being invalidate because project bc8eab398ee643eebae35cfb79ca4f7e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81432) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 12:01:50.152202 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-1b3387de-590b-4539-a302-ef32dd099767 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:50.153593 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-1b3387de-590b-4539-a302-ef32dd099767 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:50.159625 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 970/1934] 10.222.0.59 () {58 vars in 1233 bytes} [Tue Oct 11 12:01:49 2022] DELETE /identity/v3/projects/bc8eab398ee643eebae35cfb79ca4f7e => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:01:50.169142 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d10248-6df7-4ff1-a8a9-232b32e54174 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:50.169142 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d10248-6df7-4ff1-a8a9-232b32e54174 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:50.169142 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d10248-6df7-4ff1-a8a9-232b32e54174 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:50.212550 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-44d10248-6df7-4ff1-a8a9-232b32e54174 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:01:50.214243 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-44d10248-6df7-4ff1-a8a9-232b32e54174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:50.248982 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 965/1935] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:01:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:01:50.262316 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b23c63b-76f7-4150-96ce-9686235c1093 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:50.266281 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-3b23c63b-76f7-4150-96ce-9686235c1093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:50.323757 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 971/1936] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:01:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:01:50.339065 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6bdc88-92cc-4b66-9f3d-68af399f39b9 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:50.339329 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6bdc88-92cc-4b66-9f3d-68af399f39b9 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:50.339524 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6bdc88-92cc-4b66-9f3d-68af399f39b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:50.364254 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-8a6bdc88-92cc-4b66-9f3d-68af399f39b9 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:01:50.365548 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-8a6bdc88-92cc-4b66-9f3d-68af399f39b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:50.374206 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 972/1937] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:01:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:01:50.376156 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b23c63b-76f7-4150-96ce-9686235c1093 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:50.379134 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6bdc88-92cc-4b66-9f3d-68af399f39b9 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:50.380425 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224b3232-cfb1-4515-8299-ab70ed21b944 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e563fe174db4cc19ae6bf8162cc3d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:50.381090 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224b3232-cfb1-4515-8299-ab70ed21b944 service neutron] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:50.381365 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224b3232-cfb1-4515-8299-ab70ed21b944 service neutron] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:50.381642 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224b3232-cfb1-4515-8299-ab70ed21b944 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:50.396471 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224b3232-cfb1-4515-8299-ab70ed21b944 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:50.397904 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6bdc88-92cc-4b66-9f3d-68af399f39b9 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:50.398065 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224b3232-cfb1-4515-8299-ab70ed21b944 service neutron] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:50.405403 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5aceec3-0546-43e9-8e36-97b5751391d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:50.406352 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aceec3-0546-43e9-8e36-97b5751391d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:50.406777 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aceec3-0546-43e9-8e36-97b5751391d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:50.407282 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aceec3-0546-43e9-8e36-97b5751391d4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:50.408130 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5aceec3-0546-43e9-8e36-97b5751391d4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 12:01:50.408506 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5aceec3-0546-43e9-8e36-97b5751391d4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:50.410673 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5aceec3-0546-43e9-8e36-97b5751391d4 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:50.411101 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 966/1938] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:01:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 11 12:01:50.436262 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 973/1939] 10.222.0.59 () {58 vars in 1153 bytes} [Tue Oct 11 12:01:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 12:01:50.436985 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224b3232-cfb1-4515-8299-ab70ed21b944 service neutron] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:50.447463 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224b3232-cfb1-4515-8299-ab70ed21b944 service neutron] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:50.463281 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14aad731-146a-42a2-b284-19a18b0fe179 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:50.463967 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aad731-146a-42a2-b284-19a18b0fe179 admin admin] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:50.464342 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aad731-146a-42a2-b284-19a18b0fe179 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:50.464716 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aad731-146a-42a2-b284-19a18b0fe179 admin admin] PATH_INFO: `/v3/projects` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:50.465574 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14aad731-146a-42a2-b284-19a18b0fe179 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:50.467770 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14aad731-146a-42a2-b284-19a18b0fe179 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:50.488688 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:01:50.518389 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 967/1940] 10.222.0.59 () {60 vars in 1153 bytes} [Tue Oct 11 12:01:50 2022] POST /identity/v3/projects => generated 365 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 12:01:50.520417 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5aceec3-0546-43e9-8e36-97b5751391d4 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:50.542189 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5aceec3-0546-43e9-8e36-97b5751391d4 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:50.628880 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:01:50.646917 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622406b8-ec71-4a82-968c-f37b592f1d82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:50.648408 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622406b8-ec71-4a82-968c-f37b592f1d82 admin admin] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:50.649177 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622406b8-ec71-4a82-968c-f37b592f1d82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:50.649784 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622406b8-ec71-4a82-968c-f37b592f1d82 admin admin] PATH_INFO: `/v3/users` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:50.651715 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622406b8-ec71-4a82-968c-f37b592f1d82 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:50.656063 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622406b8-ec71-4a82-968c-f37b592f1d82 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:50.697277 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 974/1941] 10.222.0.59 () {60 vars in 1144 bytes} [Tue Oct 11 12:01:50 2022] POST /identity/v3/users => generated 283 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 12:01:50.698655 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14aad731-146a-42a2-b284-19a18b0fe179 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:50.716496 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14aad731-146a-42a2-b284-19a18b0fe179 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:50.723148 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b587de5a-ca28-41ac-b674-710dd11ecbc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:50.724269 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b587de5a-ca28-41ac-b674-710dd11ecbc2 admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:50.724680 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b587de5a-ca28-41ac-b674-710dd11ecbc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:50.726042 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b587de5a-ca28-41ac-b674-710dd11ecbc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:50.726231 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b587de5a-ca28-41ac-b674-710dd11ecbc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:50.729400 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b587de5a-ca28-41ac-b674-710dd11ecbc2 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:50.750687 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 968/1942] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:01:50 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:01:50.754685 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622406b8-ec71-4a82-968c-f37b592f1d82 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:50.773119 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622406b8-ec71-4a82-968c-f37b592f1d82 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:50.788290 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecec1fd-a9ac-4045-ad3b-034b86aea646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:50.789592 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecec1fd-a9ac-4045-ad3b-034b86aea646 admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:50.790256 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecec1fd-a9ac-4045-ad3b-034b86aea646 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:50.790806 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecec1fd-a9ac-4045-ad3b-034b86aea646 admin admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:50.793308 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecec1fd-a9ac-4045-ad3b-034b86aea646 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:50.798069 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecec1fd-a9ac-4045-ad3b-034b86aea646 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:50.851334 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 975/1943] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:01:50 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:01:50.852629 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b587de5a-ca28-41ac-b674-710dd11ecbc2 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:50.866354 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b587de5a-ca28-41ac-b674-710dd11ecbc2 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:50.878160 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d7dcc2-c3e4-48f8-b2aa-8fdd2a712d0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:50.879333 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d7dcc2-c3e4-48f8-b2aa-8fdd2a712d0a admin admin] REQUEST_METHOD: `PUT` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:50.879723 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d7dcc2-c3e4-48f8-b2aa-8fdd2a712d0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:50.880102 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d7dcc2-c3e4-48f8-b2aa-8fdd2a712d0a admin admin] PATH_INFO: `/v3/projects/e91c156eaffc43f7bd17da1c787481b5/users/a1d31a2575ed4deeb46e56fe25940fb6/roles/de2d2bd26fff4748a79990f97a107164` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:50.928511 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d7dcc2-c3e4-48f8-b2aa-8fdd2a712d0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e91c156eaffc43f7bd17da1c787481b5, user_id=a1d31a2575ed4deeb46e56fe25940fb6, role_id=de2d2bd26fff4748a79990f97a107164)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:50.931300 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d7dcc2-c3e4-48f8-b2aa-8fdd2a712d0a admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:50.954126 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 969/1944] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 12:01:50 2022] PUT /identity/v3/projects/e91c156eaffc43f7bd17da1c787481b5/users/a1d31a2575ed4deeb46e56fe25940fb6/roles/de2d2bd26fff4748a79990f97a107164 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:01:50.976170 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eefe263-38b0-47d5-9ce8-28fdcdd3ee59 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:50.976170 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eefe263-38b0-47d5-9ce8-28fdcdd3ee59 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:50.976472 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eefe263-38b0-47d5-9ce8-28fdcdd3ee59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:51.045382 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-8eefe263-38b0-47d5-9ce8-28fdcdd3ee59 None None] MFA Rules not processed for user `a1d31a2575ed4deeb46e56fe25940fb6`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:01:51.047266 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-8eefe263-38b0-47d5-9ce8-28fdcdd3ee59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:51.219397 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 976/1945] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:01:50 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:01:51.224536 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d7dcc2-c3e4-48f8-b2aa-8fdd2a712d0a admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:51.386987 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d7dcc2-c3e4-48f8-b2aa-8fdd2a712d0a admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:51.392672 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a865e7e-fb52-45a8-aef7-9d89f79a9bbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:51.393497 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a865e7e-fb52-45a8-aef7-9d89f79a9bbc admin admin] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:51.393753 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a865e7e-fb52-45a8-aef7-9d89f79a9bbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:51.394038 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a865e7e-fb52-45a8-aef7-9d89f79a9bbc admin admin] PATH_INFO: `/v3/projects` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:51.394761 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a865e7e-fb52-45a8-aef7-9d89f79a9bbc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:51.398302 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a865e7e-fb52-45a8-aef7-9d89f79a9bbc admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:51.454155 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 970/1946] 10.222.0.59 () {60 vars in 1153 bytes} [Tue Oct 11 12:01:51 2022] POST /identity/v3/projects => generated 363 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 12:01:51.459305 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eefe263-38b0-47d5-9ce8-28fdcdd3ee59 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:51.482541 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eefe263-38b0-47d5-9ce8-28fdcdd3ee59 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:51.625469 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94929d9d-516c-411f-99e5-9cc623202c6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:51.627459 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94929d9d-516c-411f-99e5-9cc623202c6a admin admin] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:51.628358 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94929d9d-516c-411f-99e5-9cc623202c6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:51.629033 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94929d9d-516c-411f-99e5-9cc623202c6a admin admin] PATH_INFO: `/v3/users` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:51.630723 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94929d9d-516c-411f-99e5-9cc623202c6a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:51.634317 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94929d9d-516c-411f-99e5-9cc623202c6a admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:51.637635 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a865e7e-fb52-45a8-aef7-9d89f79a9bbc admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:51.639654 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-8a865e7e-fb52-45a8-aef7-9d89f79a9bbc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:51.660549 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a865e7e-fb52-45a8-aef7-9d89f79a9bbc admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:51.670854 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e95a55f-166d-4f3e-8e25-ce1d10b61a4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:51.673207 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e95a55f-166d-4f3e-8e25-ce1d10b61a4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:51.673804 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e95a55f-166d-4f3e-8e25-ce1d10b61a4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:51.674286 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e95a55f-166d-4f3e-8e25-ce1d10b61a4c admin admin] PATH_INFO: `/v3/projects/cf3edfdac63b433d9685f678621e30f0` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:51.674977 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 977/1947] 10.222.0.59 () {60 vars in 1144 bytes} [Tue Oct 11 12:01:51 2022] POST /identity/v3/users => generated 288 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:01:51.676644 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e95a55f-166d-4f3e-8e25-ce1d10b61a4c admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf3edfdac63b433d9685f678621e30f0)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:51.680166 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94929d9d-516c-411f-99e5-9cc623202c6a admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:51.695862 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e95a55f-166d-4f3e-8e25-ce1d10b61a4c admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:51.703978 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94929d9d-516c-411f-99e5-9cc623202c6a admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:51.710656 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c53251f-3c41-4bb7-aeaa-7afd65f9b8e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:51.711517 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c53251f-3c41-4bb7-aeaa-7afd65f9b8e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:51.712023 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c53251f-3c41-4bb7-aeaa-7afd65f9b8e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:51.712394 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c53251f-3c41-4bb7-aeaa-7afd65f9b8e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:51.713268 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c53251f-3c41-4bb7-aeaa-7afd65f9b8e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:51.728900 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c53251f-3c41-4bb7-aeaa-7afd65f9b8e5 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:51.750318 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 978/1948] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:01:51 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:01:51.752272 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c53251f-3c41-4bb7-aeaa-7afd65f9b8e5 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:51.767776 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c53251f-3c41-4bb7-aeaa-7afd65f9b8e5 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:51.778121 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8af280-e080-4ffc-8ec7-0d559903cd36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:51.779078 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8af280-e080-4ffc-8ec7-0d559903cd36 admin admin] REQUEST_METHOD: `PUT` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:51.779557 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8af280-e080-4ffc-8ec7-0d559903cd36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:51.779909 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8af280-e080-4ffc-8ec7-0d559903cd36 admin admin] PATH_INFO: `/v3/projects/de444ee9068e4606afab44f6de882050/users/b19764ff573e4c2983cebc6e153daba4/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:51.799903 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8af280-e080-4ffc-8ec7-0d559903cd36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de444ee9068e4606afab44f6de882050, user_id=b19764ff573e4c2983cebc6e153daba4, role_id=5cf1f4bd2fef46ab9c6eff56d52f9f69)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:51.804194 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8af280-e080-4ffc-8ec7-0d559903cd36 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:51.814530 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-6e95a55f-166d-4f3e-8e25-ce1d10b61a4c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf3edfdac63b433d9685f678621e30f0', 'request_id': 'req-6e95a55f-166d-4f3e-8e25-ce1d10b61a4c'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:51.821947 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-6e95a55f-166d-4f3e-8e25-ce1d10b61a4c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf3edfdac63b433d9685f678621e30f0', 'request_id': 'req-6e95a55f-166d-4f3e-8e25-ce1d10b61a4c'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:51.832865 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 979/1949] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 12:01:51 2022] PUT /identity/v3/projects/de444ee9068e4606afab44f6de882050/users/b19764ff573e4c2983cebc6e153daba4/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:01:51.833278 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-6e95a55f-166d-4f3e-8e25-ce1d10b61a4c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf3edfdac63b433d9685f678621e30f0', 'request_id': 'req-6e95a55f-166d-4f3e-8e25-ce1d10b61a4c'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:51.837816 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8af280-e080-4ffc-8ec7-0d559903cd36 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:51.855312 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-6e95a55f-166d-4f3e-8e25-ce1d10b61a4c admin admin] The token cache is being invalidate because project cf3edfdac63b433d9685f678621e30f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 12:01:51.855747 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-6e95a55f-166d-4f3e-8e25-ce1d10b61a4c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:51.857140 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-6e95a55f-166d-4f3e-8e25-ce1d10b61a4c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:51.859409 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 971/1950] 10.222.0.59 () {58 vars in 1233 bytes} [Tue Oct 11 12:01:51 2022] DELETE /identity/v3/projects/cf3edfdac63b433d9685f678621e30f0 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:01:51.869498 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963c99fa-7591-4f5f-b002-a1e64a3cbb31 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:51.869916 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963c99fa-7591-4f5f-b002-a1e64a3cbb31 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:51.870389 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:01:51.870911 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963c99fa-7591-4f5f-b002-a1e64a3cbb31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:51.916745 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-963c99fa-7591-4f5f-b002-a1e64a3cbb31 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:01:51.921457 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-963c99fa-7591-4f5f-b002-a1e64a3cbb31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:51.941777 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8af280-e080-4ffc-8ec7-0d559903cd36 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:51.947077 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea39b8b-f08f-47b9-9263-eeba64244070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:51.947865 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea39b8b-f08f-47b9-9263-eeba64244070 admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:51.948167 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea39b8b-f08f-47b9-9263-eeba64244070 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:51.948515 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea39b8b-f08f-47b9-9263-eeba64244070 admin admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:51.949323 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea39b8b-f08f-47b9-9263-eeba64244070 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:51.951137 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea39b8b-f08f-47b9-9263-eeba64244070 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:51.956494 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:01:51.973353 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 980/1951] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:01:51 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 11 12:01:51.982327 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea39b8b-f08f-47b9-9263-eeba64244070 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:51.985599 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-1ea39b8b-f08f-47b9-9263-eeba64244070 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:52.012372 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea39b8b-f08f-47b9-9263-eeba64244070 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:52.022939 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0e7787-74f1-40fb-b4a2-b85942279ee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:52.024153 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0e7787-74f1-40fb-b4a2-b85942279ee4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:52.024724 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0e7787-74f1-40fb-b4a2-b85942279ee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:52.025262 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0e7787-74f1-40fb-b4a2-b85942279ee4 admin admin] PATH_INFO: `/v3/projects/de444ee9068e4606afab44f6de882050/users/b19764ff573e4c2983cebc6e153daba4/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:52.030321 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0e7787-74f1-40fb-b4a2-b85942279ee4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de444ee9068e4606afab44f6de882050, user_id=b19764ff573e4c2983cebc6e153daba4, role_id=5cf1f4bd2fef46ab9c6eff56d52f9f69)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:52.037127 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0e7787-74f1-40fb-b4a2-b85942279ee4 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:52.053166 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 981/1952] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 12:01:51 2022] PUT /identity/v3/projects/de444ee9068e4606afab44f6de882050/users/b19764ff573e4c2983cebc6e153daba4/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:01:52.056373 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 972/1953] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:01:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:01:52.064821 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c418c3-4060-41d3-a914-d19a39485062 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:52.065028 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9132f68-b2d7-40ef-819c-1c7122c069aa None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:52.065145 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9132f68-b2d7-40ef-819c-1c7122c069aa None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:52.065382 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9132f68-b2d7-40ef-819c-1c7122c069aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:52.065905 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c418c3-4060-41d3-a914-d19a39485062 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:52.066522 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c418c3-4060-41d3-a914-d19a39485062 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:52.101095 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-a9132f68-b2d7-40ef-819c-1c7122c069aa None None] MFA Rules not processed for user `b19764ff573e4c2983cebc6e153daba4`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:01:52.102669 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-a9132f68-b2d7-40ef-819c-1c7122c069aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:52.112011 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-63c418c3-4060-41d3-a914-d19a39485062 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:01:52.115656 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-63c418c3-4060-41d3-a914-d19a39485062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:52.209819 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 982/1954] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:01:52 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:01:52.216406 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 973/1955] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:01:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:01:52.221809 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9132f68-b2d7-40ef-819c-1c7122c069aa None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:52.230794 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c418c3-4060-41d3-a914-d19a39485062 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:52.234206 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9132f68-b2d7-40ef-819c-1c7122c069aa None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:52.234750 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-63c418c3-4060-41d3-a914-d19a39485062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:52.244119 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6208dea-0bd0-44d3-b8c5-9f0282d0017a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:52.244881 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6208dea-0bd0-44d3-b8c5-9f0282d0017a admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:52.245156 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6208dea-0bd0-44d3-b8c5-9f0282d0017a admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:52.245423 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6208dea-0bd0-44d3-b8c5-9f0282d0017a admin admin] PATH_INFO: `/v3/domains` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:52.246236 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6208dea-0bd0-44d3-b8c5-9f0282d0017a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 12:01:52.246544 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6208dea-0bd0-44d3-b8c5-9f0282d0017a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:52.248228 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6208dea-0bd0-44d3-b8c5-9f0282d0017a admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:52.270437 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 983/1956] 10.222.0.59 () {58 vars in 1153 bytes} [Tue Oct 11 12:01:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 12:01:52.271809 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6208dea-0bd0-44d3-b8c5-9f0282d0017a admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:52.288173 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6208dea-0bd0-44d3-b8c5-9f0282d0017a admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:52.293760 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530d33c4-f92c-4efe-9718-9a69611b2a3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:52.294396 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530d33c4-f92c-4efe-9718-9a69611b2a3e admin admin] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:52.294722 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530d33c4-f92c-4efe-9718-9a69611b2a3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:52.294992 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530d33c4-f92c-4efe-9718-9a69611b2a3e admin admin] PATH_INFO: `/v3/projects` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:52.295900 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530d33c4-f92c-4efe-9718-9a69611b2a3e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:52.297982 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530d33c4-f92c-4efe-9718-9a69611b2a3e admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:52.334182 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c418c3-4060-41d3-a914-d19a39485062 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:52.341109 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3be47fb-0ed4-4de1-b320-7b1d43396a4a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b1b651c73664d08accc6edf2f79bace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:52.342754 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3be47fb-0ed4-4de1-b320-7b1d43396a4a service cinder] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:52.343590 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3be47fb-0ed4-4de1-b320-7b1d43396a4a service cinder] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:52.344260 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3be47fb-0ed4-4de1-b320-7b1d43396a4a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:52.357267 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 984/1957] 10.222.0.59 () {60 vars in 1153 bytes} [Tue Oct 11 12:01:52 2022] POST /identity/v3/projects => generated 377 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 12:01:52.362062 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530d33c4-f92c-4efe-9718-9a69611b2a3e admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:52.362395 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3be47fb-0ed4-4de1-b320-7b1d43396a4a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:52.364108 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3be47fb-0ed4-4de1-b320-7b1d43396a4a service cinder] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:52.379351 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 974/1958] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:01:52 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:01:52.455309 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:01:52.492349 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530d33c4-f92c-4efe-9718-9a69611b2a3e admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:52.506029 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716a10d8-d70b-4e5e-94d6-c83ba321e9bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:52.506856 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716a10d8-d70b-4e5e-94d6-c83ba321e9bb admin admin] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:52.507410 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716a10d8-d70b-4e5e-94d6-c83ba321e9bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:52.507805 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716a10d8-d70b-4e5e-94d6-c83ba321e9bb admin admin] PATH_INFO: `/v3/users` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:52.508552 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716a10d8-d70b-4e5e-94d6-c83ba321e9bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:52.510717 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716a10d8-d70b-4e5e-94d6-c83ba321e9bb admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:52.559462 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3be47fb-0ed4-4de1-b320-7b1d43396a4a service cinder] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:52.569860 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 985/1959] 10.222.0.59 () {60 vars in 1144 bytes} [Tue Oct 11 12:01:52 2022] POST /identity/v3/users => generated 289 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:01:52.574666 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3be47fb-0ed4-4de1-b320-7b1d43396a4a service cinder] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:52.589265 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d41d5e-e1b2-4fa2-9dfa-9ff3a9ee20db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:52.589890 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d41d5e-e1b2-4fa2-9dfa-9ff3a9ee20db admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:52.590374 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d41d5e-e1b2-4fa2-9dfa-9ff3a9ee20db admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:52.590896 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d41d5e-e1b2-4fa2-9dfa-9ff3a9ee20db admin admin] PATH_INFO: `/v3/roles` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:52.592319 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d41d5e-e1b2-4fa2-9dfa-9ff3a9ee20db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:52.602257 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d41d5e-e1b2-4fa2-9dfa-9ff3a9ee20db admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:52.603279 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716a10d8-d70b-4e5e-94d6-c83ba321e9bb admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:52.606186 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-716a10d8-d70b-4e5e-94d6-c83ba321e9bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:52.624928 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 975/1960] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:01:52 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:01:52.631582 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d41d5e-e1b2-4fa2-9dfa-9ff3a9ee20db admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:52.659691 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d41d5e-e1b2-4fa2-9dfa-9ff3a9ee20db admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:52.664674 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fadb84-53d2-47ea-8d6b-e76f9d175229 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:52.665302 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fadb84-53d2-47ea-8d6b-e76f9d175229 admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:52.666268 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fadb84-53d2-47ea-8d6b-e76f9d175229 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:52.666268 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fadb84-53d2-47ea-8d6b-e76f9d175229 admin admin] PATH_INFO: `/v3/roles` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:52.666546 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fadb84-53d2-47ea-8d6b-e76f9d175229 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:52.668453 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fadb84-53d2-47ea-8d6b-e76f9d175229 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:52.689137 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 976/1961] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:01:52 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:01:52.693315 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fadb84-53d2-47ea-8d6b-e76f9d175229 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:52.715533 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fadb84-53d2-47ea-8d6b-e76f9d175229 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:52.732167 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e809c5ac-0e8e-4819-8bea-62d1511e00f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:52.733077 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e809c5ac-0e8e-4819-8bea-62d1511e00f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:52.733361 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e809c5ac-0e8e-4819-8bea-62d1511e00f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:52.734763 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e809c5ac-0e8e-4819-8bea-62d1511e00f8 admin admin] PATH_INFO: `/v3/projects/d34b71cc687c49a58b22c294af99523d/users/a0ce7d11d7a14c688f417dbe1315d446/roles/de2d2bd26fff4748a79990f97a107164` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:52.757431 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e809c5ac-0e8e-4819-8bea-62d1511e00f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d34b71cc687c49a58b22c294af99523d, user_id=a0ce7d11d7a14c688f417dbe1315d446, role_id=de2d2bd26fff4748a79990f97a107164)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:52.760672 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e809c5ac-0e8e-4819-8bea-62d1511e00f8 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:52.762819 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716a10d8-d70b-4e5e-94d6-c83ba321e9bb admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:52.774038 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c044244-74bd-4d4a-a7c7-ae66660b355e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b1b651c73664d08accc6edf2f79bace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:52.776748 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c044244-74bd-4d4a-a7c7-ae66660b355e service cinder] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:52.777653 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c044244-74bd-4d4a-a7c7-ae66660b355e service cinder] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:52.777951 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c044244-74bd-4d4a-a7c7-ae66660b355e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:52.782276 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-1c044244-74bd-4d4a-a7c7-ae66660b355e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:52.788297 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 977/1962] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 12:01:52 2022] PUT /identity/v3/projects/d34b71cc687c49a58b22c294af99523d/users/a0ce7d11d7a14c688f417dbe1315d446/roles/de2d2bd26fff4748a79990f97a107164 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:01:52.789514 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e809c5ac-0e8e-4819-8bea-62d1511e00f8 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:52.792796 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-e809c5ac-0e8e-4819-8bea-62d1511e00f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:52.864280 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:01:52.896240 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c044244-74bd-4d4a-a7c7-ae66660b355e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:52.898209 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c044244-74bd-4d4a-a7c7-ae66660b355e service cinder] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:52.909867 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 986/1963] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:01:52 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 11 12:01:52.911648 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89df2230-e893-4541-b5e4-2c62327fb32d None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:52.912067 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89df2230-e893-4541-b5e4-2c62327fb32d None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:52.912531 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89df2230-e893-4541-b5e4-2c62327fb32d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:52.940088 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e809c5ac-0e8e-4819-8bea-62d1511e00f8 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:52.946758 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7bc28c-5a70-4282-998d-737c0ff551bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:52.947730 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7bc28c-5a70-4282-998d-737c0ff551bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:52.947905 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7bc28c-5a70-4282-998d-737c0ff551bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:52.948115 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7bc28c-5a70-4282-998d-737c0ff551bc admin admin] PATH_INFO: `/v3/users/b0479a55cbdf487e977b8a9c51e5bfec` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:52.950630 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7bc28c-5a70-4282-998d-737c0ff551bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0479a55cbdf487e977b8a9c51e5bfec)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:52.952297 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7bc28c-5a70-4282-998d-737c0ff551bc admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:52.966336 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-89df2230-e893-4541-b5e4-2c62327fb32d None None] MFA Rules not processed for user `a0ce7d11d7a14c688f417dbe1315d446`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:01:52.967785 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-89df2230-e893-4541-b5e4-2c62327fb32d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:53.028054 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-cb7bc28c-5a70-4282-998d-737c0ff551bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0479a55cbdf487e977b8a9c51e5bfec'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:53.049783 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-cb7bc28c-5a70-4282-998d-737c0ff551bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0479a55cbdf487e977b8a9c51e5bfec'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:53.058988 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-cb7bc28c-5a70-4282-998d-737c0ff551bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0479a55cbdf487e977b8a9c51e5bfec'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:53.059958 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-cb7bc28c-5a70-4282-998d-737c0ff551bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0479a55cbdf487e977b8a9c51e5bfec'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:53.070865 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-cb7bc28c-5a70-4282-998d-737c0ff551bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0479a55cbdf487e977b8a9c51e5bfec'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:53.078499 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 978/1964] 10.222.0.59 () {58 vars in 1224 bytes} [Tue Oct 11 12:01:52 2022] DELETE /identity/v3/users/b0479a55cbdf487e977b8a9c51e5bfec => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:01:53.083293 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f82192-f0c7-4cad-aad5-a41723eb413d None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:53.083719 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f82192-f0c7-4cad-aad5-a41723eb413d None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:53.084125 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f82192-f0c7-4cad-aad5-a41723eb413d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:53.091409 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 987/1965] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:01:52 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:01:53.117591 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-60f82192-f0c7-4cad-aad5-a41723eb413d None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:01:53.120444 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-60f82192-f0c7-4cad-aad5-a41723eb413d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:53.158588 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89df2230-e893-4541-b5e4-2c62327fb32d None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:53.246417 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:01:53.285347 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 979/1966] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:01:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:01:53.319439 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f82192-f0c7-4cad-aad5-a41723eb413d None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:53.323550 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-60f82192-f0c7-4cad-aad5-a41723eb413d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:53.345713 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89df2230-e893-4541-b5e4-2c62327fb32d None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:53.411831 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:01:53.482409 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f82192-f0c7-4cad-aad5-a41723eb413d None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:53.486510 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f99dab-2c3a-435c-9c01-fc113dd4e0e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e563fe174db4cc19ae6bf8162cc3d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:53.487190 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f99dab-2c3a-435c-9c01-fc113dd4e0e3 service neutron] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:53.487468 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f99dab-2c3a-435c-9c01-fc113dd4e0e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:53.487754 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f99dab-2c3a-435c-9c01-fc113dd4e0e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:53.490249 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474d92c8-2368-4f29-a64b-b8716a9fb6ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b1b651c73664d08accc6edf2f79bace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:53.491242 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474d92c8-2368-4f29-a64b-b8716a9fb6ba service cinder] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:53.492175 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474d92c8-2368-4f29-a64b-b8716a9fb6ba service cinder] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:53.492749 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474d92c8-2368-4f29-a64b-b8716a9fb6ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:53.496130 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-474d92c8-2368-4f29-a64b-b8716a9fb6ba service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:53.499935 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f99dab-2c3a-435c-9c01-fc113dd4e0e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:53.501800 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f99dab-2c3a-435c-9c01-fc113dd4e0e3 service neutron] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:53.514499 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 980/1967] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:01:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:01:53.656512 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474d92c8-2368-4f29-a64b-b8716a9fb6ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:53.660163 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474d92c8-2368-4f29-a64b-b8716a9fb6ba service cinder] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:53.689481 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 988/1968] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:01:53 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 532 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:01:54.715015 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f99dab-2c3a-435c-9c01-fc113dd4e0e3 service neutron] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:54.717395 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-69f99dab-2c3a-435c-9c01-fc113dd4e0e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:54.738226 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f99dab-2c3a-435c-9c01-fc113dd4e0e3 service neutron] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:54.754825 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6935157-b15b-467c-a15b-d4f6ce48bb0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:54.758739 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6935157-b15b-467c-a15b-d4f6ce48bb0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:54.759551 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6935157-b15b-467c-a15b-d4f6ce48bb0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:54.760361 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6935157-b15b-467c-a15b-d4f6ce48bb0f admin admin] PATH_INFO: `/v3/projects/d703dc582b574190a930b8b2c702d705` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:54.762606 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6935157-b15b-467c-a15b-d4f6ce48bb0f admin admin] RBAC: Authorizing `identity:delete_project(project_id=d703dc582b574190a930b8b2c702d705)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:54.768042 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6935157-b15b-467c-a15b-d4f6ce48bb0f admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:54.882305 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-a6935157-b15b-467c-a15b-d4f6ce48bb0f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd703dc582b574190a930b8b2c702d705', 'request_id': 'req-a6935157-b15b-467c-a15b-d4f6ce48bb0f'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:54.891112 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-a6935157-b15b-467c-a15b-d4f6ce48bb0f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd703dc582b574190a930b8b2c702d705', 'request_id': 'req-a6935157-b15b-467c-a15b-d4f6ce48bb0f'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:54.910209 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-a6935157-b15b-467c-a15b-d4f6ce48bb0f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd703dc582b574190a930b8b2c702d705', 'request_id': 'req-a6935157-b15b-467c-a15b-d4f6ce48bb0f'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:54.939607 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-a6935157-b15b-467c-a15b-d4f6ce48bb0f admin admin] The token cache is being invalidate because project d703dc582b574190a930b8b2c702d705 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 12:01:54.939907 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-a6935157-b15b-467c-a15b-d4f6ce48bb0f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:54.941604 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-a6935157-b15b-467c-a15b-d4f6ce48bb0f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:54.943760 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 981/1969] 10.222.0.59 () {58 vars in 1233 bytes} [Tue Oct 11 12:01:54 2022] DELETE /identity/v3/projects/d703dc582b574190a930b8b2c702d705 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:01:54.969604 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474d92c8-2368-4f29-a64b-b8716a9fb6ba service cinder] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:55.004999 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474d92c8-2368-4f29-a64b-b8716a9fb6ba service cinder] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:55.121528 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e80aeb7-fb78-4869-9fcb-88918be54cbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:55.122393 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e80aeb7-fb78-4869-9fcb-88918be54cbb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:55.122806 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e80aeb7-fb78-4869-9fcb-88918be54cbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:55.123186 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e80aeb7-fb78-4869-9fcb-88918be54cbb admin admin] PATH_INFO: `/v3/users/2fd2508bd91346d29af1bed2ff856448` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:55.124721 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e80aeb7-fb78-4869-9fcb-88918be54cbb admin admin] RBAC: Authorizing `identity:delete_user(user_id=2fd2508bd91346d29af1bed2ff856448)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:55.127344 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e80aeb7-fb78-4869-9fcb-88918be54cbb admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:55.181809 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:01:55.212041 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-6e80aeb7-fb78-4869-9fcb-88918be54cbb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2fd2508bd91346d29af1bed2ff856448'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:55.247496 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-6e80aeb7-fb78-4869-9fcb-88918be54cbb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2fd2508bd91346d29af1bed2ff856448'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:55.247496 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-6e80aeb7-fb78-4869-9fcb-88918be54cbb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2fd2508bd91346d29af1bed2ff856448'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:55.259415 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-6e80aeb7-fb78-4869-9fcb-88918be54cbb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2fd2508bd91346d29af1bed2ff856448'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:55.260146 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-6e80aeb7-fb78-4869-9fcb-88918be54cbb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2fd2508bd91346d29af1bed2ff856448'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:55.274760 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 989/1970] 10.222.0.59 () {58 vars in 1224 bytes} [Tue Oct 11 12:01:54 2022] DELETE /identity/v3/users/2fd2508bd91346d29af1bed2ff856448 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:01:55.706165 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6935157-b15b-467c-a15b-d4f6ce48bb0f admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:55.707899 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-a6935157-b15b-467c-a15b-d4f6ce48bb0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:55.737571 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6935157-b15b-467c-a15b-d4f6ce48bb0f admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:55.874229 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1645d298-8698-44c1-997e-0d1f48d244cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:55.875035 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1645d298-8698-44c1-997e-0d1f48d244cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:55.875339 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1645d298-8698-44c1-997e-0d1f48d244cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:55.875603 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1645d298-8698-44c1-997e-0d1f48d244cc admin admin] PATH_INFO: `/v3/projects/edc4e6ba289f4cc2865930d616c0fd20` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:55.876831 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1645d298-8698-44c1-997e-0d1f48d244cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=edc4e6ba289f4cc2865930d616c0fd20)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:55.878953 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1645d298-8698-44c1-997e-0d1f48d244cc admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:55.987651 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-1645d298-8698-44c1-997e-0d1f48d244cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'edc4e6ba289f4cc2865930d616c0fd20', 'request_id': 'req-1645d298-8698-44c1-997e-0d1f48d244cc'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:56.004063 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-1645d298-8698-44c1-997e-0d1f48d244cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'edc4e6ba289f4cc2865930d616c0fd20', 'request_id': 'req-1645d298-8698-44c1-997e-0d1f48d244cc'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:56.015742 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-1645d298-8698-44c1-997e-0d1f48d244cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'edc4e6ba289f4cc2865930d616c0fd20', 'request_id': 'req-1645d298-8698-44c1-997e-0d1f48d244cc'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:56.045442 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:01:56.060911 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-1645d298-8698-44c1-997e-0d1f48d244cc admin admin] The token cache is being invalidate because project edc4e6ba289f4cc2865930d616c0fd20 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 12:01:56.063158 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-1645d298-8698-44c1-997e-0d1f48d244cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:56.066367 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-1645d298-8698-44c1-997e-0d1f48d244cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:56.072054 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 982/1971] 10.222.0.59 () {58 vars in 1233 bytes} [Tue Oct 11 12:01:55 2022] DELETE /identity/v3/projects/edc4e6ba289f4cc2865930d616c0fd20 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:01:56.079624 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e80aeb7-fb78-4869-9fcb-88918be54cbb admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:56.085034 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-6e80aeb7-fb78-4869-9fcb-88918be54cbb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:56.144835 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e80aeb7-fb78-4869-9fcb-88918be54cbb admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:56.298089 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34395a64-a741-4089-a884-25134c06bbd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:56.298922 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34395a64-a741-4089-a884-25134c06bbd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:56.299280 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34395a64-a741-4089-a884-25134c06bbd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:56.299740 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34395a64-a741-4089-a884-25134c06bbd0 admin admin] PATH_INFO: `/v3/users/d48cd86c413743afb8713a3789d987ed` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:56.305267 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34395a64-a741-4089-a884-25134c06bbd0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d48cd86c413743afb8713a3789d987ed)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:56.305599 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34395a64-a741-4089-a884-25134c06bbd0 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:56.432591 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-34395a64-a741-4089-a884-25134c06bbd0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd48cd86c413743afb8713a3789d987ed'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:56.476114 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-34395a64-a741-4089-a884-25134c06bbd0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd48cd86c413743afb8713a3789d987ed'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:56.477014 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-34395a64-a741-4089-a884-25134c06bbd0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd48cd86c413743afb8713a3789d987ed'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:56.491820 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-34395a64-a741-4089-a884-25134c06bbd0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd48cd86c413743afb8713a3789d987ed'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:56.492577 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-34395a64-a741-4089-a884-25134c06bbd0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd48cd86c413743afb8713a3789d987ed'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:56.526890 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 990/1972] 10.222.0.59 () {58 vars in 1224 bytes} [Tue Oct 11 12:01:56 2022] DELETE /identity/v3/users/d48cd86c413743afb8713a3789d987ed => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:01:57.120824 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1645d298-8698-44c1-997e-0d1f48d244cc admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:57.131964 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-1645d298-8698-44c1-997e-0d1f48d244cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:57.186469 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1645d298-8698-44c1-997e-0d1f48d244cc admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:57.411255 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c66ce1e-8760-49d1-86c7-1abdefe7ccf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:57.412442 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c66ce1e-8760-49d1-86c7-1abdefe7ccf7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:57.412873 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c66ce1e-8760-49d1-86c7-1abdefe7ccf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:57.413419 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c66ce1e-8760-49d1-86c7-1abdefe7ccf7 admin admin] PATH_INFO: `/v3/users/d1619ed83ae546a7ac2219c055ca4076` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:57.416285 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c66ce1e-8760-49d1-86c7-1abdefe7ccf7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1619ed83ae546a7ac2219c055ca4076)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:57.422214 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c66ce1e-8760-49d1-86c7-1abdefe7ccf7 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:57.507266 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-4c66ce1e-8760-49d1-86c7-1abdefe7ccf7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1619ed83ae546a7ac2219c055ca4076'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:57.537145 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-4c66ce1e-8760-49d1-86c7-1abdefe7ccf7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1619ed83ae546a7ac2219c055ca4076'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:57.552253 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-4c66ce1e-8760-49d1-86c7-1abdefe7ccf7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1619ed83ae546a7ac2219c055ca4076'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:57.553230 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-4c66ce1e-8760-49d1-86c7-1abdefe7ccf7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1619ed83ae546a7ac2219c055ca4076'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:57.567005 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-4c66ce1e-8760-49d1-86c7-1abdefe7ccf7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1619ed83ae546a7ac2219c055ca4076'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:57.580766 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 983/1973] 10.222.0.59 () {58 vars in 1224 bytes} [Tue Oct 11 12:01:57 2022] DELETE /identity/v3/users/d1619ed83ae546a7ac2219c055ca4076 => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:01:57.750577 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34395a64-a741-4089-a884-25134c06bbd0 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:57.753367 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-34395a64-a741-4089-a884-25134c06bbd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:57.773444 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34395a64-a741-4089-a884-25134c06bbd0 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:57.829363 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:01:57.884184 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c66ce1e-8760-49d1-86c7-1abdefe7ccf7 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:57.886121 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-4c66ce1e-8760-49d1-86c7-1abdefe7ccf7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:57.890814 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa44fd35-0f01-4b98-80c2-6a24065db013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:57.893432 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa44fd35-0f01-4b98-80c2-6a24065db013 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:57.894903 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa44fd35-0f01-4b98-80c2-6a24065db013 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:57.895374 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa44fd35-0f01-4b98-80c2-6a24065db013 admin admin] PATH_INFO: `/v3/projects/97a53cd24ef847b6952d8344fb87168b` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:57.896907 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa44fd35-0f01-4b98-80c2-6a24065db013 admin admin] RBAC: Authorizing `identity:delete_project(project_id=97a53cd24ef847b6952d8344fb87168b)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:57.901741 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa44fd35-0f01-4b98-80c2-6a24065db013 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:57.911808 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c66ce1e-8760-49d1-86c7-1abdefe7ccf7 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:57.918348 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0797d29a-d758-4c46-8946-394d294a2b1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:57.919192 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0797d29a-d758-4c46-8946-394d294a2b1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:57.919695 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0797d29a-d758-4c46-8946-394d294a2b1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:57.920715 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0797d29a-d758-4c46-8946-394d294a2b1d admin admin] PATH_INFO: `/v3/projects/bbb66bfd69114f489c0bced30cc1b0b9` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:57.922296 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0797d29a-d758-4c46-8946-394d294a2b1d admin admin] RBAC: Authorizing `identity:delete_project(project_id=bbb66bfd69114f489c0bced30cc1b0b9)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:57.924467 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0797d29a-d758-4c46-8946-394d294a2b1d admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:57.992989 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-0797d29a-d758-4c46-8946-394d294a2b1d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bbb66bfd69114f489c0bced30cc1b0b9', 'request_id': 'req-0797d29a-d758-4c46-8946-394d294a2b1d'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:57.999983 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-aa44fd35-0f01-4b98-80c2-6a24065db013 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97a53cd24ef847b6952d8344fb87168b', 'request_id': 'req-aa44fd35-0f01-4b98-80c2-6a24065db013'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:58.001952 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-0797d29a-d758-4c46-8946-394d294a2b1d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bbb66bfd69114f489c0bced30cc1b0b9', 'request_id': 'req-0797d29a-d758-4c46-8946-394d294a2b1d'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:58.011722 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-0797d29a-d758-4c46-8946-394d294a2b1d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bbb66bfd69114f489c0bced30cc1b0b9', 'request_id': 'req-0797d29a-d758-4c46-8946-394d294a2b1d'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:58.012408 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-aa44fd35-0f01-4b98-80c2-6a24065db013 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97a53cd24ef847b6952d8344fb87168b', 'request_id': 'req-aa44fd35-0f01-4b98-80c2-6a24065db013'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:58.022744 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-aa44fd35-0f01-4b98-80c2-6a24065db013 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97a53cd24ef847b6952d8344fb87168b', 'request_id': 'req-aa44fd35-0f01-4b98-80c2-6a24065db013'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:58.023424 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:01:58.029867 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-0797d29a-d758-4c46-8946-394d294a2b1d admin admin] The token cache is being invalidate because project bbb66bfd69114f489c0bced30cc1b0b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 12:01:58.031012 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-0797d29a-d758-4c46-8946-394d294a2b1d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:58.032217 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-0797d29a-d758-4c46-8946-394d294a2b1d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:58.034397 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 984/1974] 10.222.0.59 () {58 vars in 1233 bytes} [Tue Oct 11 12:01:57 2022] DELETE /identity/v3/projects/bbb66bfd69114f489c0bced30cc1b0b9 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:01:58.040313 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-aa44fd35-0f01-4b98-80c2-6a24065db013 admin admin] The token cache is being invalidate because project 97a53cd24ef847b6952d8344fb87168b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81432) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 12:01:58.040699 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-aa44fd35-0f01-4b98-80c2-6a24065db013 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:58.041646 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-aa44fd35-0f01-4b98-80c2-6a24065db013 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:01:58.043258 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047b7922-5452-40a9-a995-76c8c5ac3075 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:58.043409 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 991/1975] 10.222.0.59 () {58 vars in 1233 bytes} [Tue Oct 11 12:01:57 2022] DELETE /identity/v3/projects/97a53cd24ef847b6952d8344fb87168b => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:01:58.043546 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047b7922-5452-40a9-a995-76c8c5ac3075 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:58.043680 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047b7922-5452-40a9-a995-76c8c5ac3075 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:58.088021 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-047b7922-5452-40a9-a995-76c8c5ac3075 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:01:58.089555 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-047b7922-5452-40a9-a995-76c8c5ac3075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:58.195641 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 985/1976] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:01:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:01:58.207376 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa64d00-43f9-44c1-8995-2f6a49dd7465 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:58.207991 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa64d00-43f9-44c1-8995-2f6a49dd7465 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:58.208554 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa64d00-43f9-44c1-8995-2f6a49dd7465 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:58.249668 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-baa64d00-43f9-44c1-8995-2f6a49dd7465 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:01:58.252381 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-baa64d00-43f9-44c1-8995-2f6a49dd7465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:58.271064 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 992/1977] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:01:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:01:58.275105 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047b7922-5452-40a9-a995-76c8c5ac3075 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:58.291077 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047b7922-5452-40a9-a995-76c8c5ac3075 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:58.296075 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16b7e1b-ca95-4d7d-8c8f-d116e45b5109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:58.296853 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16b7e1b-ca95-4d7d-8c8f-d116e45b5109 admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:58.297195 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16b7e1b-ca95-4d7d-8c8f-d116e45b5109 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:58.297535 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16b7e1b-ca95-4d7d-8c8f-d116e45b5109 admin admin] PATH_INFO: `/v3/domains` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:58.298278 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16b7e1b-ca95-4d7d-8c8f-d116e45b5109 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 12:01:58.298759 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16b7e1b-ca95-4d7d-8c8f-d116e45b5109 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:58.300542 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16b7e1b-ca95-4d7d-8c8f-d116e45b5109 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:58.315254 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 986/1978] 10.222.0.59 () {58 vars in 1153 bytes} [Tue Oct 11 12:01:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 12:01:58.319889 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa64d00-43f9-44c1-8995-2f6a49dd7465 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:58.330044 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa64d00-43f9-44c1-8995-2f6a49dd7465 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:58.335102 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63b7040-6efb-4ef2-a809-8c77dced8d93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:58.335734 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63b7040-6efb-4ef2-a809-8c77dced8d93 admin admin] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:58.336010 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63b7040-6efb-4ef2-a809-8c77dced8d93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:58.336284 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63b7040-6efb-4ef2-a809-8c77dced8d93 admin admin] PATH_INFO: `/v3/projects` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:58.336933 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63b7040-6efb-4ef2-a809-8c77dced8d93 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:58.338900 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63b7040-6efb-4ef2-a809-8c77dced8d93 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:58.379361 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 993/1979] 10.222.0.59 () {60 vars in 1153 bytes} [Tue Oct 11 12:01:58 2022] POST /identity/v3/projects => generated 377 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 12:01:58.384267 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16b7e1b-ca95-4d7d-8c8f-d116e45b5109 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:58.400825 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16b7e1b-ca95-4d7d-8c8f-d116e45b5109 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:58.482112 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bea4b2b-a629-4c4b-978c-4639ce058798 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:58.483170 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bea4b2b-a629-4c4b-978c-4639ce058798 admin admin] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:58.483545 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bea4b2b-a629-4c4b-978c-4639ce058798 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:58.483951 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bea4b2b-a629-4c4b-978c-4639ce058798 admin admin] PATH_INFO: `/v3/users` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:58.484700 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bea4b2b-a629-4c4b-978c-4639ce058798 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:58.486858 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bea4b2b-a629-4c4b-978c-4639ce058798 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:58.514612 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 987/1980] 10.222.0.59 () {60 vars in 1144 bytes} [Tue Oct 11 12:01:58 2022] POST /identity/v3/users => generated 289 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:01:58.519146 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63b7040-6efb-4ef2-a809-8c77dced8d93 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:58.538294 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63b7040-6efb-4ef2-a809-8c77dced8d93 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:58.546860 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d7849c-f9dd-4636-8899-58b7420764f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:58.547878 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d7849c-f9dd-4636-8899-58b7420764f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:58.548330 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d7849c-f9dd-4636-8899-58b7420764f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:58.548833 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d7849c-f9dd-4636-8899-58b7420764f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:58.550066 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d7849c-f9dd-4636-8899-58b7420764f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:58.553537 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d7849c-f9dd-4636-8899-58b7420764f3 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:58.574200 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 994/1981] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:01:58 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 11 12:01:58.578746 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bea4b2b-a629-4c4b-978c-4639ce058798 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:58.591818 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bea4b2b-a629-4c4b-978c-4639ce058798 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:58.597691 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18e2aa4-d650-44e1-9439-987ba1257ad4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:58.598490 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18e2aa4-d650-44e1-9439-987ba1257ad4 admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:58.598893 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18e2aa4-d650-44e1-9439-987ba1257ad4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:58.599259 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18e2aa4-d650-44e1-9439-987ba1257ad4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:58.600097 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18e2aa4-d650-44e1-9439-987ba1257ad4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:58.601874 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18e2aa4-d650-44e1-9439-987ba1257ad4 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:58.615030 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 988/1982] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:01:58 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 11 12:01:58.619597 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d7849c-f9dd-4636-8899-58b7420764f3 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:58.631100 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d7849c-f9dd-4636-8899-58b7420764f3 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:58.637498 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7488ec74-cf32-45ac-b2c2-8bfd40c742e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:58.638316 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7488ec74-cf32-45ac-b2c2-8bfd40c742e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:58.638788 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7488ec74-cf32-45ac-b2c2-8bfd40c742e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:58.639070 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7488ec74-cf32-45ac-b2c2-8bfd40c742e3 admin admin] PATH_INFO: `/v3/projects/9a7beb3a9d2b4ee9ba323d5bccc618c2/users/a78e40d4eb554ca49eeec530a7adf428/roles/de2d2bd26fff4748a79990f97a107164` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:58.662137 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7488ec74-cf32-45ac-b2c2-8bfd40c742e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a7beb3a9d2b4ee9ba323d5bccc618c2, user_id=a78e40d4eb554ca49eeec530a7adf428, role_id=de2d2bd26fff4748a79990f97a107164)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:58.665681 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7488ec74-cf32-45ac-b2c2-8bfd40c742e3 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:58.688667 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 995/1983] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 12:01:58 2022] PUT /identity/v3/projects/9a7beb3a9d2b4ee9ba323d5bccc618c2/users/a78e40d4eb554ca49eeec530a7adf428/roles/de2d2bd26fff4748a79990f97a107164 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:01:58.695505 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffffda2-bf36-4a84-8165-d23d7aa78435 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:58.696023 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffffda2-bf36-4a84-8165-d23d7aa78435 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:58.696522 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffffda2-bf36-4a84-8165-d23d7aa78435 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:58.721722 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-9ffffda2-bf36-4a84-8165-d23d7aa78435 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:01:58.723187 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-9ffffda2-bf36-4a84-8165-d23d7aa78435 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:58.805069 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 989/1984] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:01:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:01:58.819328 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7488ec74-cf32-45ac-b2c2-8bfd40c742e3 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:01:58.821358 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-7488ec74-cf32-45ac-b2c2-8bfd40c742e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:01:58.892718 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7488ec74-cf32-45ac-b2c2-8bfd40c742e3 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:01:58.897018 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c41dd0-a827-4a99-af1a-0f06dc430d11 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e563fe174db4cc19ae6bf8162cc3d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:01:58.897657 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c41dd0-a827-4a99-af1a-0f06dc430d11 service neutron] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:01:58.897936 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c41dd0-a827-4a99-af1a-0f06dc430d11 service neutron] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:01:58.898210 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c41dd0-a827-4a99-af1a-0f06dc430d11 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:01:58.909132 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c41dd0-a827-4a99-af1a-0f06dc430d11 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:01:58.910666 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c41dd0-a827-4a99-af1a-0f06dc430d11 service neutron] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:01:58.916588 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:01:58.926007 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 996/1985] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:01:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:02:05.241278 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5332687a-2909-43f9-aa09-9ba21d101c8e None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:05.242194 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5332687a-2909-43f9-aa09-9ba21d101c8e None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:05.242715 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5332687a-2909-43f9-aa09-9ba21d101c8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:05.275497 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:02:05.346888 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-5332687a-2909-43f9-aa09-9ba21d101c8e None None] MFA Rules not processed for user `a78e40d4eb554ca49eeec530a7adf428`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:02:05.346888 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-5332687a-2909-43f9-aa09-9ba21d101c8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:02:05.470988 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 990/1986] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:02:05 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:02:05.492410 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c41dd0-a827-4a99-af1a-0f06dc430d11 service neutron] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:02:05.494929 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-29c41dd0-a827-4a99-af1a-0f06dc430d11 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:02:05.604626 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c41dd0-a827-4a99-af1a-0f06dc430d11 service neutron] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:02:05.613996 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130ff985-4675-4c53-a36f-7857e76bd4d8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bad5e1cb306b46818096940acc3810b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:02:05.615944 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130ff985-4675-4c53-a36f-7857e76bd4d8 service nova] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:05.616800 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130ff985-4675-4c53-a36f-7857e76bd4d8 service nova] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:05.617591 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130ff985-4675-4c53-a36f-7857e76bd4d8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:05.632003 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130ff985-4675-4c53-a36f-7857e76bd4d8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:02:05.634002 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130ff985-4675-4c53-a36f-7857e76bd4d8 service nova] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:02:05.646606 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 997/1987] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:02:05 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:02:08.498202 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf868954-8008-4db0-91a1-69b0f1597e0c None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:08.498992 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf868954-8008-4db0-91a1-69b0f1597e0c None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:08.498992 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf868954-8008-4db0-91a1-69b0f1597e0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:08.536631 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-bf868954-8008-4db0-91a1-69b0f1597e0c None None] MFA Rules not processed for user `a78e40d4eb554ca49eeec530a7adf428`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:02:08.539904 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-bf868954-8008-4db0-91a1-69b0f1597e0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:02:08.554216 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 991/1988] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:02:08 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:02:15.642141 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=34 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:02:17.688935 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130ff985-4675-4c53-a36f-7857e76bd4d8 service nova] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:02:17.691125 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-130ff985-4675-4c53-a36f-7857e76bd4d8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:02:17.706999 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130ff985-4675-4c53-a36f-7857e76bd4d8 service nova] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:02:17.712242 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff9f59a-b96f-49ad-9056-a76a9039aa08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:02:17.713908 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff9f59a-b96f-49ad-9056-a76a9039aa08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:17.714432 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff9f59a-b96f-49ad-9056-a76a9039aa08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:17.715079 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff9f59a-b96f-49ad-9056-a76a9039aa08 admin admin] PATH_INFO: `/v3/users/a1d31a2575ed4deeb46e56fe25940fb6` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:17.717083 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff9f59a-b96f-49ad-9056-a76a9039aa08 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1d31a2575ed4deeb46e56fe25940fb6)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:02:17.721106 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff9f59a-b96f-49ad-9056-a76a9039aa08 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:02:17.793368 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-3ff9f59a-b96f-49ad-9056-a76a9039aa08 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1d31a2575ed4deeb46e56fe25940fb6'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:17.824674 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-3ff9f59a-b96f-49ad-9056-a76a9039aa08 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1d31a2575ed4deeb46e56fe25940fb6'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:17.825457 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-3ff9f59a-b96f-49ad-9056-a76a9039aa08 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1d31a2575ed4deeb46e56fe25940fb6'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:17.837617 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-3ff9f59a-b96f-49ad-9056-a76a9039aa08 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1d31a2575ed4deeb46e56fe25940fb6'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:17.838539 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-3ff9f59a-b96f-49ad-9056-a76a9039aa08 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1d31a2575ed4deeb46e56fe25940fb6'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:17.855522 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 998/1989] 10.222.0.59 () {58 vars in 1224 bytes} [Tue Oct 11 12:02:17 2022] DELETE /identity/v3/users/a1d31a2575ed4deeb46e56fe25940fb6 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:02:17.860411 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88d27e4-7150-468a-9651-a4edc87a1074 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:17.861196 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88d27e4-7150-468a-9651-a4edc87a1074 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:17.861196 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88d27e4-7150-468a-9651-a4edc87a1074 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:17.887384 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-e88d27e4-7150-468a-9651-a4edc87a1074 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:02:17.889407 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-e88d27e4-7150-468a-9651-a4edc87a1074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:02:18.021861 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 992/1990] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:02:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:02:18.033109 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff9f59a-b96f-49ad-9056-a76a9039aa08 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:02:18.035727 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-3ff9f59a-b96f-49ad-9056-a76a9039aa08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:02:18.130595 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff9f59a-b96f-49ad-9056-a76a9039aa08 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:02:18.137045 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0079eca9-6932-45c9-aba4-7427adb05591 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e563fe174db4cc19ae6bf8162cc3d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:02:18.138033 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0079eca9-6932-45c9-aba4-7427adb05591 service neutron] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:18.138499 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0079eca9-6932-45c9-aba4-7427adb05591 service neutron] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:18.138947 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0079eca9-6932-45c9-aba4-7427adb05591 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:18.153630 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0079eca9-6932-45c9-aba4-7427adb05591 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:02:18.160881 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0079eca9-6932-45c9-aba4-7427adb05591 service neutron] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:02:18.181940 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 999/1991] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:02:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 11 12:02:18.704121 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88d27e4-7150-468a-9651-a4edc87a1074 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:02:18.707675 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-e88d27e4-7150-468a-9651-a4edc87a1074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:02:18.724523 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88d27e4-7150-468a-9651-a4edc87a1074 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:02:18.732909 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc40d09-d191-49fa-ba8e-c753fec3b7d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:02:18.734240 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc40d09-d191-49fa-ba8e-c753fec3b7d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:18.734853 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc40d09-d191-49fa-ba8e-c753fec3b7d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:18.735721 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc40d09-d191-49fa-ba8e-c753fec3b7d1 admin admin] PATH_INFO: `/v3/projects/e91c156eaffc43f7bd17da1c787481b5` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:18.737626 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc40d09-d191-49fa-ba8e-c753fec3b7d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e91c156eaffc43f7bd17da1c787481b5)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:02:18.741388 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc40d09-d191-49fa-ba8e-c753fec3b7d1 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:02:18.821106 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-4dc40d09-d191-49fa-ba8e-c753fec3b7d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e91c156eaffc43f7bd17da1c787481b5', 'request_id': 'req-4dc40d09-d191-49fa-ba8e-c753fec3b7d1'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:18.832890 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-4dc40d09-d191-49fa-ba8e-c753fec3b7d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e91c156eaffc43f7bd17da1c787481b5', 'request_id': 'req-4dc40d09-d191-49fa-ba8e-c753fec3b7d1'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:18.850426 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-4dc40d09-d191-49fa-ba8e-c753fec3b7d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e91c156eaffc43f7bd17da1c787481b5', 'request_id': 'req-4dc40d09-d191-49fa-ba8e-c753fec3b7d1'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:18.882652 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-4dc40d09-d191-49fa-ba8e-c753fec3b7d1 admin admin] The token cache is being invalidate because project e91c156eaffc43f7bd17da1c787481b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 12:02:18.884606 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-4dc40d09-d191-49fa-ba8e-c753fec3b7d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:18.885978 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-4dc40d09-d191-49fa-ba8e-c753fec3b7d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:18.889854 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 993/1992] 10.222.0.59 () {58 vars in 1233 bytes} [Tue Oct 11 12:02:18 2022] DELETE /identity/v3/projects/e91c156eaffc43f7bd17da1c787481b5 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:02:18.896129 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0079eca9-6932-45c9-aba4-7427adb05591 service neutron] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:02:18.914987 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0079eca9-6932-45c9-aba4-7427adb05591 service neutron] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:02:19.029764 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46145094-edd4-4aea-ba9a-27d132668403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:02:19.030277 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46145094-edd4-4aea-ba9a-27d132668403 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:19.030857 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46145094-edd4-4aea-ba9a-27d132668403 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:19.031333 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46145094-edd4-4aea-ba9a-27d132668403 admin admin] PATH_INFO: `/v3/users/b19764ff573e4c2983cebc6e153daba4` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:19.033527 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46145094-edd4-4aea-ba9a-27d132668403 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b19764ff573e4c2983cebc6e153daba4)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:02:19.036951 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46145094-edd4-4aea-ba9a-27d132668403 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:02:19.065550 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:02:19.135679 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-46145094-edd4-4aea-ba9a-27d132668403 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b19764ff573e4c2983cebc6e153daba4'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:19.158800 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-46145094-edd4-4aea-ba9a-27d132668403 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b19764ff573e4c2983cebc6e153daba4'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:19.159792 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-46145094-edd4-4aea-ba9a-27d132668403 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b19764ff573e4c2983cebc6e153daba4'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:19.173942 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-46145094-edd4-4aea-ba9a-27d132668403 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b19764ff573e4c2983cebc6e153daba4'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:19.175586 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-46145094-edd4-4aea-ba9a-27d132668403 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b19764ff573e4c2983cebc6e153daba4'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:19.206500 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1000/1993] 10.222.0.59 () {58 vars in 1224 bytes} [Tue Oct 11 12:02:18 2022] DELETE /identity/v3/users/b19764ff573e4c2983cebc6e153daba4 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:02:20.003126 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc40d09-d191-49fa-ba8e-c753fec3b7d1 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:02:20.004998 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-4dc40d09-d191-49fa-ba8e-c753fec3b7d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:02:20.021006 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc40d09-d191-49fa-ba8e-c753fec3b7d1 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:02:20.040447 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:02:20.125464 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c03161d-0c9d-4378-9fd6-2c593026c768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:02:20.126126 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c03161d-0c9d-4378-9fd6-2c593026c768 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:20.126407 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c03161d-0c9d-4378-9fd6-2c593026c768 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:20.126720 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c03161d-0c9d-4378-9fd6-2c593026c768 admin admin] PATH_INFO: `/v3/projects/de444ee9068e4606afab44f6de882050` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:20.128397 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c03161d-0c9d-4378-9fd6-2c593026c768 admin admin] RBAC: Authorizing `identity:delete_project(project_id=de444ee9068e4606afab44f6de882050)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:02:20.130872 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c03161d-0c9d-4378-9fd6-2c593026c768 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:02:20.192018 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-8c03161d-0c9d-4378-9fd6-2c593026c768 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de444ee9068e4606afab44f6de882050', 'request_id': 'req-8c03161d-0c9d-4378-9fd6-2c593026c768'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:20.200221 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-8c03161d-0c9d-4378-9fd6-2c593026c768 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de444ee9068e4606afab44f6de882050', 'request_id': 'req-8c03161d-0c9d-4378-9fd6-2c593026c768'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:20.209495 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-8c03161d-0c9d-4378-9fd6-2c593026c768 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de444ee9068e4606afab44f6de882050', 'request_id': 'req-8c03161d-0c9d-4378-9fd6-2c593026c768'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:20.223167 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-8c03161d-0c9d-4378-9fd6-2c593026c768 admin admin] The token cache is being invalidate because project de444ee9068e4606afab44f6de882050 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 12:02:20.223407 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-8c03161d-0c9d-4378-9fd6-2c593026c768 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:20.224288 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-8c03161d-0c9d-4378-9fd6-2c593026c768 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:20.225988 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 994/1994] 10.222.0.59 () {58 vars in 1233 bytes} [Tue Oct 11 12:02:20 2022] DELETE /identity/v3/projects/de444ee9068e4606afab44f6de882050 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:02:20.233566 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27f0d33-a8e9-4801-a969-80be3686d04a None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:20.234483 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27f0d33-a8e9-4801-a969-80be3686d04a None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:20.234483 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27f0d33-a8e9-4801-a969-80be3686d04a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:20.261024 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-f27f0d33-a8e9-4801-a969-80be3686d04a None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:02:20.262293 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-f27f0d33-a8e9-4801-a969-80be3686d04a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:02:20.345253 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1001/1995] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:02:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:02:20.350468 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9495cb-d9bb-4497-99ef-ccf01476ff36 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:20.350880 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9495cb-d9bb-4497-99ef-ccf01476ff36 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:20.351222 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9495cb-d9bb-4497-99ef-ccf01476ff36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:20.382769 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-dd9495cb-d9bb-4497-99ef-ccf01476ff36 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:02:20.385019 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-dd9495cb-d9bb-4497-99ef-ccf01476ff36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:02:20.399194 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 995/1996] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:02:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 12:02:20.404890 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27f0d33-a8e9-4801-a969-80be3686d04a None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:02:20.424185 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27f0d33-a8e9-4801-a969-80be3686d04a None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:02:20.432865 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fabb575-91f9-420f-845e-7213e7a981c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:02:20.433937 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fabb575-91f9-420f-845e-7213e7a981c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:20.434419 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fabb575-91f9-420f-845e-7213e7a981c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:20.434954 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fabb575-91f9-420f-845e-7213e7a981c0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:20.436236 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fabb575-91f9-420f-845e-7213e7a981c0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 12:02:20.436821 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fabb575-91f9-420f-845e-7213e7a981c0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:02:20.440298 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fabb575-91f9-420f-845e-7213e7a981c0 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:02:20.464413 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1002/1997] 10.222.0.59 () {58 vars in 1153 bytes} [Tue Oct 11 12:02:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 12:02:20.470092 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9495cb-d9bb-4497-99ef-ccf01476ff36 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:02:20.494723 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9495cb-d9bb-4497-99ef-ccf01476ff36 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:02:20.500283 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abd5115-5fa3-4e13-98ab-ed80ae2743b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:02:20.501005 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abd5115-5fa3-4e13-98ab-ed80ae2743b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:20.501380 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abd5115-5fa3-4e13-98ab-ed80ae2743b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:20.501597 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abd5115-5fa3-4e13-98ab-ed80ae2743b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:20.502200 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abd5115-5fa3-4e13-98ab-ed80ae2743b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:02:20.505041 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abd5115-5fa3-4e13-98ab-ed80ae2743b4 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:02:20.556583 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 996/1998] 10.222.0.59 () {60 vars in 1153 bytes} [Tue Oct 11 12:02:20 2022] POST /identity/v3/projects => generated 367 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:02:20.561591 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fabb575-91f9-420f-845e-7213e7a981c0 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:02:20.574109 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fabb575-91f9-420f-845e-7213e7a981c0 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:02:20.669864 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070c0b4d-371e-4dc6-9003-185c21eb65e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:02:20.670608 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070c0b4d-371e-4dc6-9003-185c21eb65e7 admin admin] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:20.670901 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070c0b4d-371e-4dc6-9003-185c21eb65e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:20.671213 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070c0b4d-371e-4dc6-9003-185c21eb65e7 admin admin] PATH_INFO: `/v3/users` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:20.671836 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070c0b4d-371e-4dc6-9003-185c21eb65e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:02:20.673899 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070c0b4d-371e-4dc6-9003-185c21eb65e7 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:02:20.705598 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1003/1999] 10.222.0.59 () {60 vars in 1144 bytes} [Tue Oct 11 12:02:20 2022] POST /identity/v3/users => generated 284 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 12:02:20.707302 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abd5115-5fa3-4e13-98ab-ed80ae2743b4 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:02:20.720296 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abd5115-5fa3-4e13-98ab-ed80ae2743b4 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:02:20.726431 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d069a7de-440f-40fa-bac0-617082d6ea5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:02:20.727231 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d069a7de-440f-40fa-bac0-617082d6ea5c admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:20.728045 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d069a7de-440f-40fa-bac0-617082d6ea5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:20.728421 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d069a7de-440f-40fa-bac0-617082d6ea5c admin admin] PATH_INFO: `/v3/roles` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:20.729257 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d069a7de-440f-40fa-bac0-617082d6ea5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:02:20.731887 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d069a7de-440f-40fa-bac0-617082d6ea5c admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:02:20.744997 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 997/2000] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:02:20 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:02:20.749707 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070c0b4d-371e-4dc6-9003-185c21eb65e7 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:02:20.767466 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070c0b4d-371e-4dc6-9003-185c21eb65e7 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:02:20.776108 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dc0284-d745-42f4-b935-699e037dbfb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:02:20.777098 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dc0284-d745-42f4-b935-699e037dbfb1 admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:20.777572 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dc0284-d745-42f4-b935-699e037dbfb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:20.778010 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dc0284-d745-42f4-b935-699e037dbfb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:20.779330 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dc0284-d745-42f4-b935-699e037dbfb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:02:20.782536 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dc0284-d745-42f4-b935-699e037dbfb1 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:02:20.792717 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:02:20.805782 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1004/2001] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:02:20 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 11 12:02:20.811162 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d069a7de-440f-40fa-bac0-617082d6ea5c admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:02:20.823654 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d069a7de-440f-40fa-bac0-617082d6ea5c admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:02:20.829277 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd58b75-5d00-4e47-aa35-830d5fd98535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:02:20.829978 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd58b75-5d00-4e47-aa35-830d5fd98535 admin admin] REQUEST_METHOD: `PUT` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:20.830264 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd58b75-5d00-4e47-aa35-830d5fd98535 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:20.830543 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd58b75-5d00-4e47-aa35-830d5fd98535 admin admin] PATH_INFO: `/v3/projects/9375aaef3bbe46bbacbfa197d82290c6/users/966c821f156345e686f8f6933d34c7b6/roles/de2d2bd26fff4748a79990f97a107164` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:20.849158 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd58b75-5d00-4e47-aa35-830d5fd98535 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9375aaef3bbe46bbacbfa197d82290c6, user_id=966c821f156345e686f8f6933d34c7b6, role_id=de2d2bd26fff4748a79990f97a107164)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:02:20.851987 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd58b75-5d00-4e47-aa35-830d5fd98535 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:02:20.868416 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 998/2002] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 12:02:20 2022] PUT /identity/v3/projects/9375aaef3bbe46bbacbfa197d82290c6/users/966c821f156345e686f8f6933d34c7b6/roles/de2d2bd26fff4748a79990f97a107164 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:02:20.881838 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97519014-06f6-462f-9ddc-6a2d840214d0 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:20.882135 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97519014-06f6-462f-9ddc-6a2d840214d0 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:20.882656 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97519014-06f6-462f-9ddc-6a2d840214d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:20.930710 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-97519014-06f6-462f-9ddc-6a2d840214d0 None None] MFA Rules not processed for user `966c821f156345e686f8f6933d34c7b6`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:02:20.932997 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-97519014-06f6-462f-9ddc-6a2d840214d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:02:21.053779 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1005/2003] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:02:20 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:02:21.059348 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd58b75-5d00-4e47-aa35-830d5fd98535 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:02:21.111025 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:02:21.136006 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd58b75-5d00-4e47-aa35-830d5fd98535 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:02:21.141688 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c27306-68c9-4760-aceb-c6d724abe5e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:02:21.142334 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c27306-68c9-4760-aceb-c6d724abe5e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:21.142660 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c27306-68c9-4760-aceb-c6d724abe5e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:21.142939 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c27306-68c9-4760-aceb-c6d724abe5e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:21.143598 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c27306-68c9-4760-aceb-c6d724abe5e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:02:21.145580 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c27306-68c9-4760-aceb-c6d724abe5e3 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:02:21.178709 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 999/2004] 10.222.0.59 () {60 vars in 1153 bytes} [Tue Oct 11 12:02:21 2022] POST /identity/v3/projects => generated 367 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:02:21.183619 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97519014-06f6-462f-9ddc-6a2d840214d0 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:02:21.196423 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97519014-06f6-462f-9ddc-6a2d840214d0 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:02:21.264664 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab955038-7571-49b1-abb7-05ad754457b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:02:21.265445 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab955038-7571-49b1-abb7-05ad754457b7 admin admin] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:21.265589 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab955038-7571-49b1-abb7-05ad754457b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:21.265793 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab955038-7571-49b1-abb7-05ad754457b7 admin admin] PATH_INFO: `/v3/users` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:21.266450 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab955038-7571-49b1-abb7-05ad754457b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:02:21.268498 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab955038-7571-49b1-abb7-05ad754457b7 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:02:21.294505 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1006/2005] 10.222.0.59 () {60 vars in 1144 bytes} [Tue Oct 11 12:02:21 2022] POST /identity/v3/users => generated 290 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:02:21.299415 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c27306-68c9-4760-aceb-c6d724abe5e3 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:02:21.312350 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c27306-68c9-4760-aceb-c6d724abe5e3 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:02:21.319384 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f99c41e-7a69-46d9-b1ad-54aa6e8374cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:02:21.320289 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f99c41e-7a69-46d9-b1ad-54aa6e8374cb admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:21.320583 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f99c41e-7a69-46d9-b1ad-54aa6e8374cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:21.320852 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f99c41e-7a69-46d9-b1ad-54aa6e8374cb admin admin] PATH_INFO: `/v3/roles` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:21.321576 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f99c41e-7a69-46d9-b1ad-54aa6e8374cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:02:21.323446 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f99c41e-7a69-46d9-b1ad-54aa6e8374cb admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:02:21.336712 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1000/2006] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:02:21 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:02:21.343120 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab955038-7571-49b1-abb7-05ad754457b7 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:02:21.354618 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab955038-7571-49b1-abb7-05ad754457b7 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:02:21.361506 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1da3897-0f3e-4e6e-84f8-13d10ff3bc97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:02:21.362753 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1da3897-0f3e-4e6e-84f8-13d10ff3bc97 admin admin] REQUEST_METHOD: `PUT` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:21.363218 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1da3897-0f3e-4e6e-84f8-13d10ff3bc97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:21.363701 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1da3897-0f3e-4e6e-84f8-13d10ff3bc97 admin admin] PATH_INFO: `/v3/projects/ad46774dcd5c420b89029b608e4fc1c0/users/08c1ae3d40d24668a3f8ac350cc199a5/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:21.392052 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1da3897-0f3e-4e6e-84f8-13d10ff3bc97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad46774dcd5c420b89029b608e4fc1c0, user_id=08c1ae3d40d24668a3f8ac350cc199a5, role_id=5cf1f4bd2fef46ab9c6eff56d52f9f69)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:02:21.395198 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1da3897-0f3e-4e6e-84f8-13d10ff3bc97 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:02:21.412807 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1007/2007] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 12:02:21 2022] PUT /identity/v3/projects/ad46774dcd5c420b89029b608e4fc1c0/users/08c1ae3d40d24668a3f8ac350cc199a5/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:02:21.417318 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f99c41e-7a69-46d9-b1ad-54aa6e8374cb admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:02:21.429480 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f99c41e-7a69-46d9-b1ad-54aa6e8374cb admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:02:21.501475 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52e9e95-b681-48f1-b00b-d5a891ccc550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:02:21.502295 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52e9e95-b681-48f1-b00b-d5a891ccc550 admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:21.502606 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52e9e95-b681-48f1-b00b-d5a891ccc550 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:21.502914 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52e9e95-b681-48f1-b00b-d5a891ccc550 admin admin] PATH_INFO: `/v3/roles` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:21.503591 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52e9e95-b681-48f1-b00b-d5a891ccc550 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:02:21.505288 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52e9e95-b681-48f1-b00b-d5a891ccc550 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:02:21.517207 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1001/2008] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:02:21 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:02:21.523827 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1da3897-0f3e-4e6e-84f8-13d10ff3bc97 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:02:21.539803 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1da3897-0f3e-4e6e-84f8-13d10ff3bc97 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:02:21.549762 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2d12d5-5776-448a-a256-340979b7a12e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:02:21.550110 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2d12d5-5776-448a-a256-340979b7a12e admin admin] REQUEST_METHOD: `PUT` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:21.550444 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2d12d5-5776-448a-a256-340979b7a12e admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:21.550959 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2d12d5-5776-448a-a256-340979b7a12e admin admin] PATH_INFO: `/v3/projects/ad46774dcd5c420b89029b608e4fc1c0/users/08c1ae3d40d24668a3f8ac350cc199a5/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:21.554528 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2d12d5-5776-448a-a256-340979b7a12e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad46774dcd5c420b89029b608e4fc1c0, user_id=08c1ae3d40d24668a3f8ac350cc199a5, role_id=5cf1f4bd2fef46ab9c6eff56d52f9f69)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:02:21.559907 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2d12d5-5776-448a-a256-340979b7a12e admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:02:21.580497 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1008/2009] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 12:02:21 2022] PUT /identity/v3/projects/ad46774dcd5c420b89029b608e4fc1c0/users/08c1ae3d40d24668a3f8ac350cc199a5/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 12:02:21.600332 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e421af59-6c8c-4da7-aefa-d02edd110a61 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:21.600813 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e421af59-6c8c-4da7-aefa-d02edd110a61 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:21.601288 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e421af59-6c8c-4da7-aefa-d02edd110a61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:21.649088 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-e421af59-6c8c-4da7-aefa-d02edd110a61 None None] MFA Rules not processed for user `08c1ae3d40d24668a3f8ac350cc199a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:02:21.650445 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-e421af59-6c8c-4da7-aefa-d02edd110a61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:02:21.766540 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1002/2010] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:02:21 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:02:21.775040 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2d12d5-5776-448a-a256-340979b7a12e admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:02:21.777763 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-0f2d12d5-5776-448a-a256-340979b7a12e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:02:21.843582 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:02:21.859161 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2d12d5-5776-448a-a256-340979b7a12e admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:02:21.863594 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c622ba6a-9d41-457c-9e9f-6fe41e282673 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b1b651c73664d08accc6edf2f79bace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:02:21.864334 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c622ba6a-9d41-457c-9e9f-6fe41e282673 service cinder] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:21.864671 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c622ba6a-9d41-457c-9e9f-6fe41e282673 service cinder] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:21.864941 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c622ba6a-9d41-457c-9e9f-6fe41e282673 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:21.877289 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c622ba6a-9d41-457c-9e9f-6fe41e282673 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:02:21.879147 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c622ba6a-9d41-457c-9e9f-6fe41e282673 service cinder] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:02:21.891553 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1009/2011] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:02:21 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 11 12:02:21.948620 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e421af59-6c8c-4da7-aefa-d02edd110a61 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:02:21.950329 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-e421af59-6c8c-4da7-aefa-d02edd110a61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:02:21.965303 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e421af59-6c8c-4da7-aefa-d02edd110a61 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:02:21.970887 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3eba372-40c5-4920-ba48-de67b002a618 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b1b651c73664d08accc6edf2f79bace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:02:21.970887 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eba372-40c5-4920-ba48-de67b002a618 service cinder] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:21.971189 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eba372-40c5-4920-ba48-de67b002a618 service cinder] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:21.971345 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eba372-40c5-4920-ba48-de67b002a618 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:22.010109 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:02:22.044809 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3eba372-40c5-4920-ba48-de67b002a618 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:02:22.046358 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3eba372-40c5-4920-ba48-de67b002a618 service cinder] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:02:22.062066 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1003/2012] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:02:21 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:02:26.084238 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c622ba6a-9d41-457c-9e9f-6fe41e282673 service cinder] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:02:26.172426 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c622ba6a-9d41-457c-9e9f-6fe41e282673 service cinder] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:02:26.177634 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3f9551-1730-4b14-bfbf-6e4dda8af01f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:02:26.178298 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3f9551-1730-4b14-bfbf-6e4dda8af01f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:26.178580 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3f9551-1730-4b14-bfbf-6e4dda8af01f admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:26.178879 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3f9551-1730-4b14-bfbf-6e4dda8af01f admin admin] PATH_INFO: `/v3/users/966c821f156345e686f8f6933d34c7b6` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:26.180082 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3f9551-1730-4b14-bfbf-6e4dda8af01f admin admin] RBAC: Authorizing `identity:delete_user(user_id=966c821f156345e686f8f6933d34c7b6)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:02:26.182069 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3f9551-1730-4b14-bfbf-6e4dda8af01f admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:02:26.241948 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-be3f9551-1730-4b14-bfbf-6e4dda8af01f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '966c821f156345e686f8f6933d34c7b6'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:26.255500 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-be3f9551-1730-4b14-bfbf-6e4dda8af01f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '966c821f156345e686f8f6933d34c7b6'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:26.256194 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-be3f9551-1730-4b14-bfbf-6e4dda8af01f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '966c821f156345e686f8f6933d34c7b6'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:26.264875 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-be3f9551-1730-4b14-bfbf-6e4dda8af01f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '966c821f156345e686f8f6933d34c7b6'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:26.265583 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-be3f9551-1730-4b14-bfbf-6e4dda8af01f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '966c821f156345e686f8f6933d34c7b6'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:26.280653 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1010/2013] 10.222.0.59 () {58 vars in 1224 bytes} [Tue Oct 11 12:02:26 2022] DELETE /identity/v3/users/966c821f156345e686f8f6933d34c7b6 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 12:02:26.285885 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091f2822-caf3-487d-b3db-9884baff38cf None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:26.286405 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091f2822-caf3-487d-b3db-9884baff38cf None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:26.286495 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091f2822-caf3-487d-b3db-9884baff38cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:26.307153 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-091f2822-caf3-487d-b3db-9884baff38cf None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:02:26.308364 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-091f2822-caf3-487d-b3db-9884baff38cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:02:26.382796 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1004/2014] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:02:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:02:26.394545 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3f9551-1730-4b14-bfbf-6e4dda8af01f admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:02:26.396745 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-be3f9551-1730-4b14-bfbf-6e4dda8af01f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:02:26.467571 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3f9551-1730-4b14-bfbf-6e4dda8af01f admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:02:26.471380 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a6f3f7-a10f-4168-9693-e2b126d09bac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e563fe174db4cc19ae6bf8162cc3d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:02:26.472009 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a6f3f7-a10f-4168-9693-e2b126d09bac service neutron] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:26.472299 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a6f3f7-a10f-4168-9693-e2b126d09bac service neutron] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:26.472569 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a6f3f7-a10f-4168-9693-e2b126d09bac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:26.488937 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a6f3f7-a10f-4168-9693-e2b126d09bac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:02:26.490488 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a6f3f7-a10f-4168-9693-e2b126d09bac service neutron] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:02:26.501587 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1011/2015] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:02:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:02:26.929859 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091f2822-caf3-487d-b3db-9884baff38cf None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:02:26.931932 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-091f2822-caf3-487d-b3db-9884baff38cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:02:26.952450 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091f2822-caf3-487d-b3db-9884baff38cf None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:02:26.973207 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf0f745-eb3a-4cc3-9462-d8b4e17f3fcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:02:26.974119 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf0f745-eb3a-4cc3-9462-d8b4e17f3fcf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:26.974656 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf0f745-eb3a-4cc3-9462-d8b4e17f3fcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:26.975138 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf0f745-eb3a-4cc3-9462-d8b4e17f3fcf admin admin] PATH_INFO: `/v3/projects/9375aaef3bbe46bbacbfa197d82290c6` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:26.977552 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf0f745-eb3a-4cc3-9462-d8b4e17f3fcf admin admin] RBAC: Authorizing `identity:delete_project(project_id=9375aaef3bbe46bbacbfa197d82290c6)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:02:26.981353 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf0f745-eb3a-4cc3-9462-d8b4e17f3fcf admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:02:27.066775 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-bdf0f745-eb3a-4cc3-9462-d8b4e17f3fcf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9375aaef3bbe46bbacbfa197d82290c6', 'request_id': 'req-bdf0f745-eb3a-4cc3-9462-d8b4e17f3fcf'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:27.074194 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-bdf0f745-eb3a-4cc3-9462-d8b4e17f3fcf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9375aaef3bbe46bbacbfa197d82290c6', 'request_id': 'req-bdf0f745-eb3a-4cc3-9462-d8b4e17f3fcf'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:27.085578 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-bdf0f745-eb3a-4cc3-9462-d8b4e17f3fcf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9375aaef3bbe46bbacbfa197d82290c6', 'request_id': 'req-bdf0f745-eb3a-4cc3-9462-d8b4e17f3fcf'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:27.108398 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-bdf0f745-eb3a-4cc3-9462-d8b4e17f3fcf admin admin] The token cache is being invalidate because project 9375aaef3bbe46bbacbfa197d82290c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 12:02:27.109670 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-bdf0f745-eb3a-4cc3-9462-d8b4e17f3fcf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:27.110948 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-bdf0f745-eb3a-4cc3-9462-d8b4e17f3fcf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:27.114006 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1005/2016] 10.222.0.59 () {58 vars in 1233 bytes} [Tue Oct 11 12:02:26 2022] DELETE /identity/v3/projects/9375aaef3bbe46bbacbfa197d82290c6 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:02:27.119886 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a6f3f7-a10f-4168-9693-e2b126d09bac service neutron] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:02:27.135653 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a6f3f7-a10f-4168-9693-e2b126d09bac service neutron] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:02:27.142974 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:02:27.263553 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3703d1a-6319-4d28-8c94-8c5be5771c9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:02:27.265886 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3703d1a-6319-4d28-8c94-8c5be5771c9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:27.266743 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3703d1a-6319-4d28-8c94-8c5be5771c9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:27.267472 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3703d1a-6319-4d28-8c94-8c5be5771c9a admin admin] PATH_INFO: `/v3/users/08c1ae3d40d24668a3f8ac350cc199a5` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:27.270369 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3703d1a-6319-4d28-8c94-8c5be5771c9a admin admin] RBAC: Authorizing `identity:delete_user(user_id=08c1ae3d40d24668a3f8ac350cc199a5)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:02:27.276515 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3703d1a-6319-4d28-8c94-8c5be5771c9a admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:02:27.370171 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-c3703d1a-6319-4d28-8c94-8c5be5771c9a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08c1ae3d40d24668a3f8ac350cc199a5'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:27.387145 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-c3703d1a-6319-4d28-8c94-8c5be5771c9a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08c1ae3d40d24668a3f8ac350cc199a5'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:27.388653 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-c3703d1a-6319-4d28-8c94-8c5be5771c9a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08c1ae3d40d24668a3f8ac350cc199a5'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:27.402275 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-c3703d1a-6319-4d28-8c94-8c5be5771c9a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08c1ae3d40d24668a3f8ac350cc199a5'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:27.407084 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-c3703d1a-6319-4d28-8c94-8c5be5771c9a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08c1ae3d40d24668a3f8ac350cc199a5'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:27.427831 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1012/2017] 10.222.0.59 () {58 vars in 1224 bytes} [Tue Oct 11 12:02:27 2022] DELETE /identity/v3/users/08c1ae3d40d24668a3f8ac350cc199a5 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:02:27.937189 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf0f745-eb3a-4cc3-9462-d8b4e17f3fcf admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:02:27.939067 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-bdf0f745-eb3a-4cc3-9462-d8b4e17f3fcf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:02:27.966448 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf0f745-eb3a-4cc3-9462-d8b4e17f3fcf admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:02:28.049965 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:02:28.062487 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b1c570-8b77-4c0e-8fbb-d1b2aec20731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:02:28.064256 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b1c570-8b77-4c0e-8fbb-d1b2aec20731 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:28.064576 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b1c570-8b77-4c0e-8fbb-d1b2aec20731 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:28.065043 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b1c570-8b77-4c0e-8fbb-d1b2aec20731 admin admin] PATH_INFO: `/v3/projects/ad46774dcd5c420b89029b608e4fc1c0` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:28.067373 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b1c570-8b77-4c0e-8fbb-d1b2aec20731 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad46774dcd5c420b89029b608e4fc1c0)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:02:28.072104 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b1c570-8b77-4c0e-8fbb-d1b2aec20731 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:02:28.155859 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-f2b1c570-8b77-4c0e-8fbb-d1b2aec20731 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad46774dcd5c420b89029b608e4fc1c0', 'request_id': 'req-f2b1c570-8b77-4c0e-8fbb-d1b2aec20731'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:28.163937 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-f2b1c570-8b77-4c0e-8fbb-d1b2aec20731 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad46774dcd5c420b89029b608e4fc1c0', 'request_id': 'req-f2b1c570-8b77-4c0e-8fbb-d1b2aec20731'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:28.173681 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-f2b1c570-8b77-4c0e-8fbb-d1b2aec20731 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad46774dcd5c420b89029b608e4fc1c0', 'request_id': 'req-f2b1c570-8b77-4c0e-8fbb-d1b2aec20731'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:28.187851 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-f2b1c570-8b77-4c0e-8fbb-d1b2aec20731 admin admin] The token cache is being invalidate because project ad46774dcd5c420b89029b608e4fc1c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 12:02:28.188175 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-f2b1c570-8b77-4c0e-8fbb-d1b2aec20731 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:28.188977 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-f2b1c570-8b77-4c0e-8fbb-d1b2aec20731 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:28.190533 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1006/2018] 10.222.0.59 () {58 vars in 1233 bytes} [Tue Oct 11 12:02:27 2022] DELETE /identity/v3/projects/ad46774dcd5c420b89029b608e4fc1c0 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:02:28.196068 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9ac9d7-dc5f-4997-8b9d-abb6d2291d74 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:28.196622 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9ac9d7-dc5f-4997-8b9d-abb6d2291d74 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:28.196622 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9ac9d7-dc5f-4997-8b9d-abb6d2291d74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:28.224715 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-2d9ac9d7-dc5f-4997-8b9d-abb6d2291d74 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:02:28.226441 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-2d9ac9d7-dc5f-4997-8b9d-abb6d2291d74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:02:28.341374 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1013/2019] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:02:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 12:02:28.346221 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131a45a6-4cd1-4852-a2e2-dea2211b964f None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:28.346504 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131a45a6-4cd1-4852-a2e2-dea2211b964f None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:28.346803 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131a45a6-4cd1-4852-a2e2-dea2211b964f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:28.379501 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-131a45a6-4cd1-4852-a2e2-dea2211b964f None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:02:28.382655 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-131a45a6-4cd1-4852-a2e2-dea2211b964f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:02:28.398540 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1007/2020] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:02:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:02:28.404991 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9ac9d7-dc5f-4997-8b9d-abb6d2291d74 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:02:28.425631 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9ac9d7-dc5f-4997-8b9d-abb6d2291d74 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:02:28.433661 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdc4ae3-3980-41a6-b6e9-13ec99d5895c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:02:28.434699 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdc4ae3-3980-41a6-b6e9-13ec99d5895c admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:28.435161 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdc4ae3-3980-41a6-b6e9-13ec99d5895c admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:28.435622 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdc4ae3-3980-41a6-b6e9-13ec99d5895c admin admin] PATH_INFO: `/v3/domains` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:28.436794 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdc4ae3-3980-41a6-b6e9-13ec99d5895c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 12:02:28.437558 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdc4ae3-3980-41a6-b6e9-13ec99d5895c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:02:28.440387 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdc4ae3-3980-41a6-b6e9-13ec99d5895c admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:02:28.468048 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1014/2021] 10.222.0.59 () {58 vars in 1153 bytes} [Tue Oct 11 12:02:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 12:02:28.469702 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131a45a6-4cd1-4852-a2e2-dea2211b964f None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:02:28.485718 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131a45a6-4cd1-4852-a2e2-dea2211b964f None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:02:28.494349 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250035b5-a3f4-479b-a0d9-c1f407893050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:02:28.494916 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250035b5-a3f4-479b-a0d9-c1f407893050 admin admin] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:28.495380 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250035b5-a3f4-479b-a0d9-c1f407893050 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:28.496677 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250035b5-a3f4-479b-a0d9-c1f407893050 admin admin] PATH_INFO: `/v3/projects` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:28.497128 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250035b5-a3f4-479b-a0d9-c1f407893050 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:02:28.500632 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250035b5-a3f4-479b-a0d9-c1f407893050 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:02:28.547380 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1008/2022] 10.222.0.59 () {60 vars in 1153 bytes} [Tue Oct 11 12:02:28 2022] POST /identity/v3/projects => generated 383 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 12:02:28.557149 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdc4ae3-3980-41a6-b6e9-13ec99d5895c admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:02:28.576787 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdc4ae3-3980-41a6-b6e9-13ec99d5895c admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:02:28.598815 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:02:28.689974 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ea0534-187f-4c31-95ed-5f1ad4b92a93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:02:28.690675 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ea0534-187f-4c31-95ed-5f1ad4b92a93 admin admin] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:28.691149 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ea0534-187f-4c31-95ed-5f1ad4b92a93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:28.691621 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ea0534-187f-4c31-95ed-5f1ad4b92a93 admin admin] PATH_INFO: `/v3/users` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:28.692679 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ea0534-187f-4c31-95ed-5f1ad4b92a93 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:02:28.695020 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ea0534-187f-4c31-95ed-5f1ad4b92a93 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:02:28.729628 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1015/2023] 10.222.0.59 () {60 vars in 1144 bytes} [Tue Oct 11 12:02:28 2022] POST /identity/v3/users => generated 292 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:02:28.736556 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250035b5-a3f4-479b-a0d9-c1f407893050 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:02:28.756570 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250035b5-a3f4-479b-a0d9-c1f407893050 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:02:28.766210 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f8b3b8-14d2-4eab-9365-e7914d7ae353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:02:28.767473 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f8b3b8-14d2-4eab-9365-e7914d7ae353 admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:28.767934 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f8b3b8-14d2-4eab-9365-e7914d7ae353 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:28.768354 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f8b3b8-14d2-4eab-9365-e7914d7ae353 admin admin] PATH_INFO: `/v3/roles` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:28.769225 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f8b3b8-14d2-4eab-9365-e7914d7ae353 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:02:28.771832 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f8b3b8-14d2-4eab-9365-e7914d7ae353 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:02:28.785515 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1009/2024] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:02:28 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:02:28.789786 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ea0534-187f-4c31-95ed-5f1ad4b92a93 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:02:28.801781 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ea0534-187f-4c31-95ed-5f1ad4b92a93 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:02:28.807060 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11aedc58-98d0-4cce-9c37-b7c1386ad485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:02:28.807699 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aedc58-98d0-4cce-9c37-b7c1386ad485 admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:28.807984 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aedc58-98d0-4cce-9c37-b7c1386ad485 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:28.808251 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aedc58-98d0-4cce-9c37-b7c1386ad485 admin admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:28.808916 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11aedc58-98d0-4cce-9c37-b7c1386ad485 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:02:28.810686 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11aedc58-98d0-4cce-9c37-b7c1386ad485 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:02:28.824919 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1016/2025] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:02:28 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 11 12:02:28.830241 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f8b3b8-14d2-4eab-9365-e7914d7ae353 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:02:28.842226 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f8b3b8-14d2-4eab-9365-e7914d7ae353 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:02:28.848474 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc69c43-1ff7-4a39-bb8a-b6d3ca4f2916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:02:28.849154 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc69c43-1ff7-4a39-bb8a-b6d3ca4f2916 admin admin] REQUEST_METHOD: `PUT` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:28.849447 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc69c43-1ff7-4a39-bb8a-b6d3ca4f2916 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:28.849729 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc69c43-1ff7-4a39-bb8a-b6d3ca4f2916 admin admin] PATH_INFO: `/v3/projects/8eaf3a485c2746b887780a2d49e7432a/users/9c03b5b2969b40819beb9c0e565f005f/roles/de2d2bd26fff4748a79990f97a107164` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:28.869915 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc69c43-1ff7-4a39-bb8a-b6d3ca4f2916 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8eaf3a485c2746b887780a2d49e7432a, user_id=9c03b5b2969b40819beb9c0e565f005f, role_id=de2d2bd26fff4748a79990f97a107164)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:02:28.872729 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc69c43-1ff7-4a39-bb8a-b6d3ca4f2916 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:02:28.894324 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1010/2026] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 12:02:28 2022] PUT /identity/v3/projects/8eaf3a485c2746b887780a2d49e7432a/users/9c03b5b2969b40819beb9c0e565f005f/roles/de2d2bd26fff4748a79990f97a107164 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:02:28.909023 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3d2e08-50a8-49de-85c2-e6ffb4cb7c86 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:28.909574 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3d2e08-50a8-49de-85c2-e6ffb4cb7c86 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:28.910062 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3d2e08-50a8-49de-85c2-e6ffb4cb7c86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:28.960208 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-9f3d2e08-50a8-49de-85c2-e6ffb4cb7c86 None None] MFA Rules not processed for user `9c03b5b2969b40819beb9c0e565f005f`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:02:28.962538 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-9f3d2e08-50a8-49de-85c2-e6ffb4cb7c86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:02:29.046517 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1017/2027] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:02:28 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:02:29.052093 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc69c43-1ff7-4a39-bb8a-b6d3ca4f2916 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:02:29.121697 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc69c43-1ff7-4a39-bb8a-b6d3ca4f2916 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:02:29.126813 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d45f6b-37c6-4f1c-be51-e7d58685993e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:02:29.127454 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d45f6b-37c6-4f1c-be51-e7d58685993e admin admin] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:29.127733 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d45f6b-37c6-4f1c-be51-e7d58685993e admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:29.128004 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d45f6b-37c6-4f1c-be51-e7d58685993e admin admin] PATH_INFO: `/v3/projects` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:29.128665 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d45f6b-37c6-4f1c-be51-e7d58685993e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:02:29.130651 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d45f6b-37c6-4f1c-be51-e7d58685993e admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:02:29.162271 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1011/2028] 10.222.0.59 () {60 vars in 1153 bytes} [Tue Oct 11 12:02:29 2022] POST /identity/v3/projects => generated 379 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:02:29.167615 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3d2e08-50a8-49de-85c2-e6ffb4cb7c86 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:02:29.180473 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3d2e08-50a8-49de-85c2-e6ffb4cb7c86 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:02:29.250466 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c667db-0e9c-4f1b-bf55-76ac952425e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:02:29.251629 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c667db-0e9c-4f1b-bf55-76ac952425e1 admin admin] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:29.252032 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c667db-0e9c-4f1b-bf55-76ac952425e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:29.252391 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c667db-0e9c-4f1b-bf55-76ac952425e1 admin admin] PATH_INFO: `/v3/users` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:29.253128 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c667db-0e9c-4f1b-bf55-76ac952425e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:02:29.255484 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c667db-0e9c-4f1b-bf55-76ac952425e1 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:02:29.285684 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1018/2029] 10.222.0.59 () {60 vars in 1144 bytes} [Tue Oct 11 12:02:29 2022] POST /identity/v3/users => generated 296 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:02:29.291844 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d45f6b-37c6-4f1c-be51-e7d58685993e admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:02:29.304593 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d45f6b-37c6-4f1c-be51-e7d58685993e admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:02:29.310726 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5530001b-054e-43cc-98bf-e54eaa1d0276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:02:29.311382 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5530001b-054e-43cc-98bf-e54eaa1d0276 admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:29.311661 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5530001b-054e-43cc-98bf-e54eaa1d0276 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:29.311932 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5530001b-054e-43cc-98bf-e54eaa1d0276 admin admin] PATH_INFO: `/v3/roles` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:29.312594 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5530001b-054e-43cc-98bf-e54eaa1d0276 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:02:29.314260 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5530001b-054e-43cc-98bf-e54eaa1d0276 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:02:29.317206 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:02:29.329028 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1012/2030] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:02:29 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:02:29.336459 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c667db-0e9c-4f1b-bf55-76ac952425e1 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:02:29.353993 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c667db-0e9c-4f1b-bf55-76ac952425e1 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:02:29.362248 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764c1af1-dc6c-45df-8c71-1c7de7ce7f56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:02:29.363449 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764c1af1-dc6c-45df-8c71-1c7de7ce7f56 admin admin] REQUEST_METHOD: `PUT` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:29.363918 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764c1af1-dc6c-45df-8c71-1c7de7ce7f56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:29.364388 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764c1af1-dc6c-45df-8c71-1c7de7ce7f56 admin admin] PATH_INFO: `/v3/projects/af215541c69b4e6086664927655e738e/users/e5b07af1671e4d7f8e46b1d831ce2a74/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:29.381445 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:02:29.396156 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764c1af1-dc6c-45df-8c71-1c7de7ce7f56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af215541c69b4e6086664927655e738e, user_id=e5b07af1671e4d7f8e46b1d831ce2a74, role_id=5cf1f4bd2fef46ab9c6eff56d52f9f69)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:02:29.399316 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764c1af1-dc6c-45df-8c71-1c7de7ce7f56 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:02:29.416530 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1019/2031] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 12:02:29 2022] PUT /identity/v3/projects/af215541c69b4e6086664927655e738e/users/e5b07af1671e4d7f8e46b1d831ce2a74/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:02:29.421473 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5530001b-054e-43cc-98bf-e54eaa1d0276 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:02:29.434187 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5530001b-054e-43cc-98bf-e54eaa1d0276 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:02:29.537182 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f34419f-4e8a-4478-856f-36a49beaf60f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:02:29.538169 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f34419f-4e8a-4478-856f-36a49beaf60f admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:29.538676 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f34419f-4e8a-4478-856f-36a49beaf60f admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:29.539127 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f34419f-4e8a-4478-856f-36a49beaf60f admin admin] PATH_INFO: `/v3/roles` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:29.540380 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f34419f-4e8a-4478-856f-36a49beaf60f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:02:29.543327 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f34419f-4e8a-4478-856f-36a49beaf60f admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:02:29.563613 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1013/2032] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:02:29 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:02:29.570120 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764c1af1-dc6c-45df-8c71-1c7de7ce7f56 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:02:29.581993 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764c1af1-dc6c-45df-8c71-1c7de7ce7f56 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:02:29.587465 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7f6ee2-e684-48ba-8309-e7be66e6a471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:02:29.588230 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7f6ee2-e684-48ba-8309-e7be66e6a471 admin admin] REQUEST_METHOD: `PUT` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:29.588508 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7f6ee2-e684-48ba-8309-e7be66e6a471 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:29.588779 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7f6ee2-e684-48ba-8309-e7be66e6a471 admin admin] PATH_INFO: `/v3/projects/af215541c69b4e6086664927655e738e/users/e5b07af1671e4d7f8e46b1d831ce2a74/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:29.591845 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7f6ee2-e684-48ba-8309-e7be66e6a471 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af215541c69b4e6086664927655e738e, user_id=e5b07af1671e4d7f8e46b1d831ce2a74, role_id=5cf1f4bd2fef46ab9c6eff56d52f9f69)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:02:29.596707 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7f6ee2-e684-48ba-8309-e7be66e6a471 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:02:29.610177 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1020/2033] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 12:02:29 2022] PUT /identity/v3/projects/af215541c69b4e6086664927655e738e/users/e5b07af1671e4d7f8e46b1d831ce2a74/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:02:29.622184 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a70fb1c-b8d7-431b-9ada-ab3457fd937d None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:29.622428 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a70fb1c-b8d7-431b-9ada-ab3457fd937d None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:29.622795 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a70fb1c-b8d7-431b-9ada-ab3457fd937d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:29.671984 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-7a70fb1c-b8d7-431b-9ada-ab3457fd937d None None] MFA Rules not processed for user `e5b07af1671e4d7f8e46b1d831ce2a74`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:02:29.673722 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-7a70fb1c-b8d7-431b-9ada-ab3457fd937d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:02:29.762659 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1014/2034] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:02:29 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 12:02:29.775833 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7f6ee2-e684-48ba-8309-e7be66e6a471 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:02:29.778003 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-2e7f6ee2-e684-48ba-8309-e7be66e6a471 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:02:29.888821 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7f6ee2-e684-48ba-8309-e7be66e6a471 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:02:29.895184 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cd9cdb-07ce-41da-8d27-7d8c9d6f8631 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b1b651c73664d08accc6edf2f79bace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:02:29.896179 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cd9cdb-07ce-41da-8d27-7d8c9d6f8631 service cinder] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:29.896604 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cd9cdb-07ce-41da-8d27-7d8c9d6f8631 service cinder] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:29.897126 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cd9cdb-07ce-41da-8d27-7d8c9d6f8631 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:29.913329 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cd9cdb-07ce-41da-8d27-7d8c9d6f8631 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:02:29.915771 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cd9cdb-07ce-41da-8d27-7d8c9d6f8631 service cinder] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:02:29.950719 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1021/2035] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:02:29 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:02:30.130579 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a70fb1c-b8d7-431b-9ada-ab3457fd937d None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:02:30.142157 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a70fb1c-b8d7-431b-9ada-ab3457fd937d None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:02:30.146761 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233d283d-c9fb-4544-81c9-33c639c86ea3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b1b651c73664d08accc6edf2f79bace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:02:30.147419 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233d283d-c9fb-4544-81c9-33c639c86ea3 service cinder] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:30.147701 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233d283d-c9fb-4544-81c9-33c639c86ea3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:30.147973 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233d283d-c9fb-4544-81c9-33c639c86ea3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:30.227234 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233d283d-c9fb-4544-81c9-33c639c86ea3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:02:30.229956 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233d283d-c9fb-4544-81c9-33c639c86ea3 service cinder] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:02:30.244299 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1015/2036] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:02:30 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:02:35.796822 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2b45ec-a381-471e-a391-eff31e33ac13 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:35.797431 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2b45ec-a381-471e-a391-eff31e33ac13 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:35.797809 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2b45ec-a381-471e-a391-eff31e33ac13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:35.825304 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-6d2b45ec-a381-471e-a391-eff31e33ac13 None None] MFA Rules not processed for user `ae372b81003e41aeb654da0bcca928d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:02:35.827604 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-6d2b45ec-a381-471e-a391-eff31e33ac13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:02:35.954867 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1022/2037] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:02:35 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:02:36.532621 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233d283d-c9fb-4544-81c9-33c639c86ea3 service cinder] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:02:36.534217 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-233d283d-c9fb-4544-81c9-33c639c86ea3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:02:36.581538 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:02:36.680013 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233d283d-c9fb-4544-81c9-33c639c86ea3 service cinder] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:02:36.689628 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d9f303-a27d-4c53-b820-11f4158b16c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:02:36.690814 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d9f303-a27d-4c53-b820-11f4158b16c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:36.691313 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d9f303-a27d-4c53-b820-11f4158b16c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:36.692234 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d9f303-a27d-4c53-b820-11f4158b16c5 admin admin] PATH_INFO: `/v3/users/a0ce7d11d7a14c688f417dbe1315d446` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:36.694946 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d9f303-a27d-4c53-b820-11f4158b16c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0ce7d11d7a14c688f417dbe1315d446)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:02:36.698786 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d9f303-a27d-4c53-b820-11f4158b16c5 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:02:36.793293 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-65d9f303-a27d-4c53-b820-11f4158b16c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0ce7d11d7a14c688f417dbe1315d446'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:36.808604 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-65d9f303-a27d-4c53-b820-11f4158b16c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0ce7d11d7a14c688f417dbe1315d446'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:36.820531 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-65d9f303-a27d-4c53-b820-11f4158b16c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0ce7d11d7a14c688f417dbe1315d446'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:36.822695 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-65d9f303-a27d-4c53-b820-11f4158b16c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0ce7d11d7a14c688f417dbe1315d446'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:36.844768 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-65d9f303-a27d-4c53-b820-11f4158b16c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0ce7d11d7a14c688f417dbe1315d446'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:36.863281 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1016/2038] 10.222.0.59 () {58 vars in 1224 bytes} [Tue Oct 11 12:02:36 2022] DELETE /identity/v3/users/a0ce7d11d7a14c688f417dbe1315d446 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:02:36.868904 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d57425-3fe8-4c44-9e70-26013316c92e None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:36.869486 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d57425-3fe8-4c44-9e70-26013316c92e None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:36.869486 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d57425-3fe8-4c44-9e70-26013316c92e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:36.892236 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-63d57425-3fe8-4c44-9e70-26013316c92e None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:02:36.893557 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-63d57425-3fe8-4c44-9e70-26013316c92e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:02:36.993962 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1023/2039] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:02:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:02:37.017369 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d9f303-a27d-4c53-b820-11f4158b16c5 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:02:37.017369 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-65d9f303-a27d-4c53-b820-11f4158b16c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:02:37.086325 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d9f303-a27d-4c53-b820-11f4158b16c5 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:02:37.091755 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1cfc59-b91b-4b29-9388-7fff9d663c56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e563fe174db4cc19ae6bf8162cc3d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:02:37.092760 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1cfc59-b91b-4b29-9388-7fff9d663c56 service neutron] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:37.092760 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1cfc59-b91b-4b29-9388-7fff9d663c56 service neutron] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:37.093218 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1cfc59-b91b-4b29-9388-7fff9d663c56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:37.104767 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1cfc59-b91b-4b29-9388-7fff9d663c56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:02:37.106321 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1cfc59-b91b-4b29-9388-7fff9d663c56 service neutron] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:02:37.118263 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1017/2040] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:02:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 11 12:02:37.617942 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d57425-3fe8-4c44-9e70-26013316c92e None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:02:37.620670 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-63d57425-3fe8-4c44-9e70-26013316c92e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:02:37.668471 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d57425-3fe8-4c44-9e70-26013316c92e None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:02:37.678170 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4237c821-d4fe-4e12-92e3-d956ee1d51b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:02:37.679254 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4237c821-d4fe-4e12-92e3-d956ee1d51b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:37.679848 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4237c821-d4fe-4e12-92e3-d956ee1d51b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:37.679848 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4237c821-d4fe-4e12-92e3-d956ee1d51b2 admin admin] PATH_INFO: `/v3/projects/d34b71cc687c49a58b22c294af99523d` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:37.681459 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4237c821-d4fe-4e12-92e3-d956ee1d51b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d34b71cc687c49a58b22c294af99523d)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:02:37.683985 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4237c821-d4fe-4e12-92e3-d956ee1d51b2 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:02:37.714827 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:02:37.768806 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-4237c821-d4fe-4e12-92e3-d956ee1d51b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd34b71cc687c49a58b22c294af99523d', 'request_id': 'req-4237c821-d4fe-4e12-92e3-d956ee1d51b2'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:37.776116 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-4237c821-d4fe-4e12-92e3-d956ee1d51b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd34b71cc687c49a58b22c294af99523d', 'request_id': 'req-4237c821-d4fe-4e12-92e3-d956ee1d51b2'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:37.784105 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-4237c821-d4fe-4e12-92e3-d956ee1d51b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd34b71cc687c49a58b22c294af99523d', 'request_id': 'req-4237c821-d4fe-4e12-92e3-d956ee1d51b2'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:37.800032 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-4237c821-d4fe-4e12-92e3-d956ee1d51b2 admin admin] The token cache is being invalidate because project d34b71cc687c49a58b22c294af99523d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81432) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 12:02:37.800369 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-4237c821-d4fe-4e12-92e3-d956ee1d51b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:37.801255 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-4237c821-d4fe-4e12-92e3-d956ee1d51b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:37.803197 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1024/2041] 10.222.0.59 () {58 vars in 1233 bytes} [Tue Oct 11 12:02:37 2022] DELETE /identity/v3/projects/d34b71cc687c49a58b22c294af99523d => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:02:37.809566 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f13f7c-8f95-47ff-ba89-b0a466bdf5f6 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:37.810228 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f13f7c-8f95-47ff-ba89-b0a466bdf5f6 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:37.810228 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f13f7c-8f95-47ff-ba89-b0a466bdf5f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:37.840554 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-b6f13f7c-8f95-47ff-ba89-b0a466bdf5f6 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:02:37.841837 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-b6f13f7c-8f95-47ff-ba89-b0a466bdf5f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:02:37.888436 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:02:37.926657 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1018/2042] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:02:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:02:37.934726 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec83802-b679-4919-9180-3212fe2c2033 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:37.935399 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec83802-b679-4919-9180-3212fe2c2033 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:37.935399 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec83802-b679-4919-9180-3212fe2c2033 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:37.958907 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-cec83802-b679-4919-9180-3212fe2c2033 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:02:37.960175 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-cec83802-b679-4919-9180-3212fe2c2033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:02:37.969181 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1025/2043] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:02:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 12:02:37.973858 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f13f7c-8f95-47ff-ba89-b0a466bdf5f6 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:02:37.987551 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f13f7c-8f95-47ff-ba89-b0a466bdf5f6 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:02:37.992699 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e88ba0-aadb-4f16-8dd2-56ee686152ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:02:37.993353 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e88ba0-aadb-4f16-8dd2-56ee686152ef admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:37.993636 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e88ba0-aadb-4f16-8dd2-56ee686152ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:37.993970 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e88ba0-aadb-4f16-8dd2-56ee686152ef admin admin] PATH_INFO: `/v3/domains` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:37.994598 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e88ba0-aadb-4f16-8dd2-56ee686152ef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 12:02:37.994901 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e88ba0-aadb-4f16-8dd2-56ee686152ef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:02:37.996557 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e88ba0-aadb-4f16-8dd2-56ee686152ef admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:02:38.011274 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1019/2044] 10.222.0.59 () {58 vars in 1153 bytes} [Tue Oct 11 12:02:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 12:02:38.016627 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec83802-b679-4919-9180-3212fe2c2033 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:02:38.035548 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec83802-b679-4919-9180-3212fe2c2033 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:02:38.044037 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062cb2d8-8b3b-4c7a-b45d-35db00299389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:02:38.045237 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062cb2d8-8b3b-4c7a-b45d-35db00299389 admin admin] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:38.045711 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062cb2d8-8b3b-4c7a-b45d-35db00299389 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:38.046198 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062cb2d8-8b3b-4c7a-b45d-35db00299389 admin admin] PATH_INFO: `/v3/projects` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:38.047605 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062cb2d8-8b3b-4c7a-b45d-35db00299389 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:02:38.052386 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062cb2d8-8b3b-4c7a-b45d-35db00299389 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:02:38.110615 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1026/2045] 10.222.0.59 () {60 vars in 1153 bytes} [Tue Oct 11 12:02:38 2022] POST /identity/v3/projects => generated 371 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:02:38.113862 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e88ba0-aadb-4f16-8dd2-56ee686152ef admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:02:38.132711 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e88ba0-aadb-4f16-8dd2-56ee686152ef admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:02:38.266309 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39749fc4-9f12-47e4-b3e1-9108e95aea7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:02:38.267763 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39749fc4-9f12-47e4-b3e1-9108e95aea7c admin admin] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:38.268238 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39749fc4-9f12-47e4-b3e1-9108e95aea7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:38.273639 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39749fc4-9f12-47e4-b3e1-9108e95aea7c admin admin] PATH_INFO: `/v3/users` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:38.275009 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39749fc4-9f12-47e4-b3e1-9108e95aea7c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:02:38.279630 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39749fc4-9f12-47e4-b3e1-9108e95aea7c admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:02:38.321253 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1020/2046] 10.222.0.59 () {60 vars in 1144 bytes} [Tue Oct 11 12:02:38 2022] POST /identity/v3/users => generated 286 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 12:02:38.326745 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062cb2d8-8b3b-4c7a-b45d-35db00299389 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:02:38.357402 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062cb2d8-8b3b-4c7a-b45d-35db00299389 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:02:38.367925 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8559cc2b-cabd-49c7-870f-6b8cdda23fb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:02:38.369005 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8559cc2b-cabd-49c7-870f-6b8cdda23fb5 admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:38.369489 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8559cc2b-cabd-49c7-870f-6b8cdda23fb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:38.369988 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8559cc2b-cabd-49c7-870f-6b8cdda23fb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:38.371232 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8559cc2b-cabd-49c7-870f-6b8cdda23fb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:02:38.375139 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8559cc2b-cabd-49c7-870f-6b8cdda23fb5 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:02:38.401916 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1027/2047] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:02:38 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:02:38.403811 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39749fc4-9f12-47e4-b3e1-9108e95aea7c admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:02:38.421679 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39749fc4-9f12-47e4-b3e1-9108e95aea7c admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:02:38.429413 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9f50ec-fc96-4617-8f3f-8ce278c27eba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:02:38.434902 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9f50ec-fc96-4617-8f3f-8ce278c27eba admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:38.435485 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9f50ec-fc96-4617-8f3f-8ce278c27eba admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:38.436028 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9f50ec-fc96-4617-8f3f-8ce278c27eba admin admin] PATH_INFO: `/v3/roles` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:38.437516 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9f50ec-fc96-4617-8f3f-8ce278c27eba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:02:38.440940 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9f50ec-fc96-4617-8f3f-8ce278c27eba admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:02:38.464154 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1021/2048] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:02:38 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 11 12:02:38.468758 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8559cc2b-cabd-49c7-870f-6b8cdda23fb5 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:02:38.480421 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8559cc2b-cabd-49c7-870f-6b8cdda23fb5 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:02:38.486516 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9cfd1f-bdf5-4fad-9512-b06a7d1bd11f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:02:38.487284 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9cfd1f-bdf5-4fad-9512-b06a7d1bd11f admin admin] REQUEST_METHOD: `PUT` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:38.487562 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9cfd1f-bdf5-4fad-9512-b06a7d1bd11f admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:38.487837 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9cfd1f-bdf5-4fad-9512-b06a7d1bd11f admin admin] PATH_INFO: `/v3/projects/3786ec273a6a48da8938e1ab7137f36e/users/ef9538e140304e018eff25f9d6f04cee/roles/de2d2bd26fff4748a79990f97a107164` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:38.510410 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9cfd1f-bdf5-4fad-9512-b06a7d1bd11f admin admin] RBAC: Authorizing `identity:create_grant(project_id=3786ec273a6a48da8938e1ab7137f36e, user_id=ef9538e140304e018eff25f9d6f04cee, role_id=de2d2bd26fff4748a79990f97a107164)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:02:38.513678 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9cfd1f-bdf5-4fad-9512-b06a7d1bd11f admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:02:38.537598 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1028/2049] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 12:02:38 2022] PUT /identity/v3/projects/3786ec273a6a48da8938e1ab7137f36e/users/ef9538e140304e018eff25f9d6f04cee/roles/de2d2bd26fff4748a79990f97a107164 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:02:38.550736 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9b4a61-09d7-4570-9bea-3e5de6dd88a7 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:38.552398 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9b4a61-09d7-4570-9bea-3e5de6dd88a7 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:38.554119 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9b4a61-09d7-4570-9bea-3e5de6dd88a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:38.615119 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-1b9b4a61-09d7-4570-9bea-3e5de6dd88a7 None None] MFA Rules not processed for user `ef9538e140304e018eff25f9d6f04cee`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:02:38.615119 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-1b9b4a61-09d7-4570-9bea-3e5de6dd88a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:02:38.648495 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9cfd1f-bdf5-4fad-9512-b06a7d1bd11f admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:02:38.651730 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-ab9cfd1f-bdf5-4fad-9512-b06a7d1bd11f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:02:38.716245 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1022/2050] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:02:38 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:02:38.729758 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9b4a61-09d7-4570-9bea-3e5de6dd88a7 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:02:38.731998 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-1b9b4a61-09d7-4570-9bea-3e5de6dd88a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:02:38.784699 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9cfd1f-bdf5-4fad-9512-b06a7d1bd11f admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:02:38.786069 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:02:38.792631 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d894540-feae-4bc3-9099-e9b68f8f20fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:02:38.793376 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d894540-feae-4bc3-9099-e9b68f8f20fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:38.793740 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d894540-feae-4bc3-9099-e9b68f8f20fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:38.794076 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d894540-feae-4bc3-9099-e9b68f8f20fc admin admin] PATH_INFO: `/v3/users/9c03b5b2969b40819beb9c0e565f005f` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:38.795735 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d894540-feae-4bc3-9099-e9b68f8f20fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c03b5b2969b40819beb9c0e565f005f)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:02:38.798357 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d894540-feae-4bc3-9099-e9b68f8f20fc admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:02:38.812832 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9b4a61-09d7-4570-9bea-3e5de6dd88a7 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:02:38.829148 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fe6b8d-94d7-497f-b45e-3caec0f29e18 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b1b651c73664d08accc6edf2f79bace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:02:38.829970 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fe6b8d-94d7-497f-b45e-3caec0f29e18 service cinder] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:38.830397 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fe6b8d-94d7-497f-b45e-3caec0f29e18 service cinder] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:38.830800 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fe6b8d-94d7-497f-b45e-3caec0f29e18 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:38.843595 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fe6b8d-94d7-497f-b45e-3caec0f29e18 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:02:38.845704 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fe6b8d-94d7-497f-b45e-3caec0f29e18 service cinder] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:02:38.857429 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1023/2051] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:02:38 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 11 12:02:38.866240 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:02:38.871149 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-1d894540-feae-4bc3-9099-e9b68f8f20fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c03b5b2969b40819beb9c0e565f005f'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:38.904175 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fe6b8d-94d7-497f-b45e-3caec0f29e18 service cinder] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:02:38.905451 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-1d894540-feae-4bc3-9099-e9b68f8f20fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c03b5b2969b40819beb9c0e565f005f'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:38.906264 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-1d894540-feae-4bc3-9099-e9b68f8f20fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c03b5b2969b40819beb9c0e565f005f'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:38.927707 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fe6b8d-94d7-497f-b45e-3caec0f29e18 service cinder] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:02:38.930123 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-1d894540-feae-4bc3-9099-e9b68f8f20fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c03b5b2969b40819beb9c0e565f005f'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:38.931201 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-1d894540-feae-4bc3-9099-e9b68f8f20fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c03b5b2969b40819beb9c0e565f005f'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:38.937868 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f7d95e-44b5-42ad-ab3f-56b3c21b82b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:02:38.939045 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f7d95e-44b5-42ad-ab3f-56b3c21b82b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:38.939479 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f7d95e-44b5-42ad-ab3f-56b3c21b82b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:38.939947 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f7d95e-44b5-42ad-ab3f-56b3c21b82b8 admin admin] PATH_INFO: `/v3/users/ef9538e140304e018eff25f9d6f04cee` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:38.941853 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f7d95e-44b5-42ad-ab3f-56b3c21b82b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef9538e140304e018eff25f9d6f04cee)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:02:38.945160 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f7d95e-44b5-42ad-ab3f-56b3c21b82b8 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:02:38.979455 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1029/2052] 10.222.0.59 () {58 vars in 1224 bytes} [Tue Oct 11 12:02:38 2022] DELETE /identity/v3/users/9c03b5b2969b40819beb9c0e565f005f => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:02:38.985490 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ccea6d-d28f-47c0-926c-e8580b740290 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:38.986036 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ccea6d-d28f-47c0-926c-e8580b740290 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:38.986404 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ccea6d-d28f-47c0-926c-e8580b740290 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:39.049248 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-04ccea6d-d28f-47c0-926c-e8580b740290 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:02:39.051364 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-04ccea6d-d28f-47c0-926c-e8580b740290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:02:39.106070 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-39f7d95e-44b5-42ad-ab3f-56b3c21b82b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef9538e140304e018eff25f9d6f04cee'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:39.156992 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-39f7d95e-44b5-42ad-ab3f-56b3c21b82b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef9538e140304e018eff25f9d6f04cee'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:39.173075 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-39f7d95e-44b5-42ad-ab3f-56b3c21b82b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef9538e140304e018eff25f9d6f04cee'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:39.174696 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-39f7d95e-44b5-42ad-ab3f-56b3c21b82b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef9538e140304e018eff25f9d6f04cee'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:39.189058 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1030/2053] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:02:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:02:39.212634 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ccea6d-d28f-47c0-926c-e8580b740290 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:02:39.220222 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-04ccea6d-d28f-47c0-926c-e8580b740290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:02:39.230737 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-39f7d95e-44b5-42ad-ab3f-56b3c21b82b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef9538e140304e018eff25f9d6f04cee'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:39.259867 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1024/2054] 10.222.0.59 () {58 vars in 1224 bytes} [Tue Oct 11 12:02:38 2022] DELETE /identity/v3/users/ef9538e140304e018eff25f9d6f04cee => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:02:39.267578 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70139b4-5a85-4f2e-8d0f-c8cb87442087 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:39.268228 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70139b4-5a85-4f2e-8d0f-c8cb87442087 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:39.268712 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70139b4-5a85-4f2e-8d0f-c8cb87442087 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:39.348142 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-d70139b4-5a85-4f2e-8d0f-c8cb87442087 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:02:39.350251 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-d70139b4-5a85-4f2e-8d0f-c8cb87442087 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:02:39.446610 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ccea6d-d28f-47c0-926c-e8580b740290 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:02:39.553974 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fe15c7-370c-46d6-9347-f9990d88efd3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e563fe174db4cc19ae6bf8162cc3d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:02:39.554614 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1025/2055] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:02:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:02:39.558315 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fe15c7-370c-46d6-9347-f9990d88efd3 service neutron] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:39.558484 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fe15c7-370c-46d6-9347-f9990d88efd3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:39.558876 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fe15c7-370c-46d6-9347-f9990d88efd3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:39.563221 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-77fe15c7-370c-46d6-9347-f9990d88efd3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:02:39.567735 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70139b4-5a85-4f2e-8d0f-c8cb87442087 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:02:39.572487 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-d70139b4-5a85-4f2e-8d0f-c8cb87442087 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:02:39.582202 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fe15c7-370c-46d6-9347-f9990d88efd3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:02:39.584125 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fe15c7-370c-46d6-9347-f9990d88efd3 service neutron] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:02:39.612113 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70139b4-5a85-4f2e-8d0f-c8cb87442087 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:02:39.619861 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1031/2056] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:02:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:02:39.627792 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213847fd-878a-4a49-a16f-54f41b2d94fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e563fe174db4cc19ae6bf8162cc3d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:02:39.632279 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213847fd-878a-4a49-a16f-54f41b2d94fe service neutron] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:39.633331 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213847fd-878a-4a49-a16f-54f41b2d94fe service neutron] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:39.635924 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213847fd-878a-4a49-a16f-54f41b2d94fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:39.690070 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213847fd-878a-4a49-a16f-54f41b2d94fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:02:39.693011 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213847fd-878a-4a49-a16f-54f41b2d94fe service neutron] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:02:39.748057 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1026/2057] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:02:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:02:40.390121 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fe15c7-370c-46d6-9347-f9990d88efd3 service neutron] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:02:40.395835 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-77fe15c7-370c-46d6-9347-f9990d88efd3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:02:40.425642 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fe15c7-370c-46d6-9347-f9990d88efd3 service neutron] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:02:40.434550 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b148ba67-04c7-49bb-a3d2-66129a8d8753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:02:40.435747 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b148ba67-04c7-49bb-a3d2-66129a8d8753 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:40.436228 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b148ba67-04c7-49bb-a3d2-66129a8d8753 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:40.436716 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b148ba67-04c7-49bb-a3d2-66129a8d8753 admin admin] PATH_INFO: `/v3/projects/3786ec273a6a48da8938e1ab7137f36e` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:40.439111 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b148ba67-04c7-49bb-a3d2-66129a8d8753 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3786ec273a6a48da8938e1ab7137f36e)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:02:40.442794 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b148ba67-04c7-49bb-a3d2-66129a8d8753 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:02:40.480178 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:02:40.537785 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-b148ba67-04c7-49bb-a3d2-66129a8d8753 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3786ec273a6a48da8938e1ab7137f36e', 'request_id': 'req-b148ba67-04c7-49bb-a3d2-66129a8d8753'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:40.552127 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-b148ba67-04c7-49bb-a3d2-66129a8d8753 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3786ec273a6a48da8938e1ab7137f36e', 'request_id': 'req-b148ba67-04c7-49bb-a3d2-66129a8d8753'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:40.578127 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-b148ba67-04c7-49bb-a3d2-66129a8d8753 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3786ec273a6a48da8938e1ab7137f36e', 'request_id': 'req-b148ba67-04c7-49bb-a3d2-66129a8d8753'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:40.618897 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-b148ba67-04c7-49bb-a3d2-66129a8d8753 admin admin] The token cache is being invalidate because project 3786ec273a6a48da8938e1ab7137f36e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81432) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 12:02:40.618897 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-b148ba67-04c7-49bb-a3d2-66129a8d8753 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:40.624342 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-b148ba67-04c7-49bb-a3d2-66129a8d8753 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:40.626528 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1032/2058] 10.222.0.59 () {58 vars in 1233 bytes} [Tue Oct 11 12:02:40 2022] DELETE /identity/v3/projects/3786ec273a6a48da8938e1ab7137f36e => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:02:41.107390 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213847fd-878a-4a49-a16f-54f41b2d94fe service neutron] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:02:41.122225 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-213847fd-878a-4a49-a16f-54f41b2d94fe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:02:41.154190 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213847fd-878a-4a49-a16f-54f41b2d94fe service neutron] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:02:41.241613 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:02:41.302606 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895a13c5-0555-4ecf-9fbb-2299c1c3521e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:02:41.303551 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895a13c5-0555-4ecf-9fbb-2299c1c3521e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:41.303978 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895a13c5-0555-4ecf-9fbb-2299c1c3521e admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:41.304439 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895a13c5-0555-4ecf-9fbb-2299c1c3521e admin admin] PATH_INFO: `/v3/projects/8eaf3a485c2746b887780a2d49e7432a` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:41.305856 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895a13c5-0555-4ecf-9fbb-2299c1c3521e admin admin] RBAC: Authorizing `identity:delete_project(project_id=8eaf3a485c2746b887780a2d49e7432a)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:02:41.308067 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895a13c5-0555-4ecf-9fbb-2299c1c3521e admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:02:41.370807 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-895a13c5-0555-4ecf-9fbb-2299c1c3521e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8eaf3a485c2746b887780a2d49e7432a', 'request_id': 'req-895a13c5-0555-4ecf-9fbb-2299c1c3521e'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:41.378022 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-895a13c5-0555-4ecf-9fbb-2299c1c3521e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8eaf3a485c2746b887780a2d49e7432a', 'request_id': 'req-895a13c5-0555-4ecf-9fbb-2299c1c3521e'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:41.389421 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-895a13c5-0555-4ecf-9fbb-2299c1c3521e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8eaf3a485c2746b887780a2d49e7432a', 'request_id': 'req-895a13c5-0555-4ecf-9fbb-2299c1c3521e'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:41.408255 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-895a13c5-0555-4ecf-9fbb-2299c1c3521e admin admin] The token cache is being invalidate because project 8eaf3a485c2746b887780a2d49e7432a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 12:02:41.408680 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-895a13c5-0555-4ecf-9fbb-2299c1c3521e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:41.409741 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-895a13c5-0555-4ecf-9fbb-2299c1c3521e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:41.412601 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1027/2059] 10.222.0.59 () {58 vars in 1233 bytes} [Tue Oct 11 12:02:41 2022] DELETE /identity/v3/projects/8eaf3a485c2746b887780a2d49e7432a => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 12:02:41.417697 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b148ba67-04c7-49bb-a3d2-66129a8d8753 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:02:41.419636 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-b148ba67-04c7-49bb-a3d2-66129a8d8753 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:02:41.441710 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b148ba67-04c7-49bb-a3d2-66129a8d8753 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:02:41.518888 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e386543f-8725-40b3-a8f1-7291071b7434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:02:41.519569 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e386543f-8725-40b3-a8f1-7291071b7434 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:41.519852 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e386543f-8725-40b3-a8f1-7291071b7434 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:41.520259 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e386543f-8725-40b3-a8f1-7291071b7434 admin admin] PATH_INFO: `/v3/users/e5b07af1671e4d7f8e46b1d831ce2a74` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:41.521888 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e386543f-8725-40b3-a8f1-7291071b7434 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5b07af1671e4d7f8e46b1d831ce2a74)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:02:41.524115 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e386543f-8725-40b3-a8f1-7291071b7434 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:02:41.600194 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-e386543f-8725-40b3-a8f1-7291071b7434 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5b07af1671e4d7f8e46b1d831ce2a74'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:41.620036 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-e386543f-8725-40b3-a8f1-7291071b7434 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5b07af1671e4d7f8e46b1d831ce2a74'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:41.620848 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-e386543f-8725-40b3-a8f1-7291071b7434 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5b07af1671e4d7f8e46b1d831ce2a74'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:41.631001 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-e386543f-8725-40b3-a8f1-7291071b7434 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5b07af1671e4d7f8e46b1d831ce2a74'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:41.631688 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-e386543f-8725-40b3-a8f1-7291071b7434 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5b07af1671e4d7f8e46b1d831ce2a74'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:41.648958 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1033/2060] 10.222.0.59 () {58 vars in 1224 bytes} [Tue Oct 11 12:02:41 2022] DELETE /identity/v3/users/e5b07af1671e4d7f8e46b1d831ce2a74 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 12:02:42.167683 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895a13c5-0555-4ecf-9fbb-2299c1c3521e admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:02:42.169764 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-895a13c5-0555-4ecf-9fbb-2299c1c3521e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:02:42.214201 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895a13c5-0555-4ecf-9fbb-2299c1c3521e admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:02:42.322509 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efeb9328-2233-4055-8ff1-28ba050e35b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:02:42.323623 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efeb9328-2233-4055-8ff1-28ba050e35b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:42.323917 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efeb9328-2233-4055-8ff1-28ba050e35b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:42.324232 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efeb9328-2233-4055-8ff1-28ba050e35b2 admin admin] PATH_INFO: `/v3/projects/af215541c69b4e6086664927655e738e` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:42.325593 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efeb9328-2233-4055-8ff1-28ba050e35b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=af215541c69b4e6086664927655e738e)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:02:42.327792 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efeb9328-2233-4055-8ff1-28ba050e35b2 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:02:42.395660 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-efeb9328-2233-4055-8ff1-28ba050e35b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af215541c69b4e6086664927655e738e', 'request_id': 'req-efeb9328-2233-4055-8ff1-28ba050e35b2'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:42.403147 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-efeb9328-2233-4055-8ff1-28ba050e35b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af215541c69b4e6086664927655e738e', 'request_id': 'req-efeb9328-2233-4055-8ff1-28ba050e35b2'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:42.412493 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-efeb9328-2233-4055-8ff1-28ba050e35b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af215541c69b4e6086664927655e738e', 'request_id': 'req-efeb9328-2233-4055-8ff1-28ba050e35b2'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:42.425290 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:02:42.432914 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-efeb9328-2233-4055-8ff1-28ba050e35b2 admin admin] The token cache is being invalidate because project af215541c69b4e6086664927655e738e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 12:02:42.433299 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-efeb9328-2233-4055-8ff1-28ba050e35b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:42.434171 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-efeb9328-2233-4055-8ff1-28ba050e35b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:42.438436 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1028/2061] 10.222.0.59 () {58 vars in 1233 bytes} [Tue Oct 11 12:02:42 2022] DELETE /identity/v3/projects/af215541c69b4e6086664927655e738e => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:02:42.444789 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301698e7-95b3-47fc-b7b6-8a64839d9d4b None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:42.445602 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301698e7-95b3-47fc-b7b6-8a64839d9d4b None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:42.445938 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301698e7-95b3-47fc-b7b6-8a64839d9d4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:42.483399 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-301698e7-95b3-47fc-b7b6-8a64839d9d4b None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:02:42.485590 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-301698e7-95b3-47fc-b7b6-8a64839d9d4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:02:42.509442 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:02:42.572846 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1034/2062] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:02:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:02:42.584745 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8319ebb5-be91-4d16-b3f7-2abd71e917ee None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:42.585400 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8319ebb5-be91-4d16-b3f7-2abd71e917ee None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:42.586338 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8319ebb5-be91-4d16-b3f7-2abd71e917ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:42.617114 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-8319ebb5-be91-4d16-b3f7-2abd71e917ee None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:02:42.618569 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-8319ebb5-be91-4d16-b3f7-2abd71e917ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:02:42.627417 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1029/2063] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:02:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 12:02:42.632476 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301698e7-95b3-47fc-b7b6-8a64839d9d4b None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:02:42.648081 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301698e7-95b3-47fc-b7b6-8a64839d9d4b None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:02:42.656531 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe927aa8-4814-4fb3-a707-95376a9dd388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:02:42.657601 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe927aa8-4814-4fb3-a707-95376a9dd388 admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:42.658085 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe927aa8-4814-4fb3-a707-95376a9dd388 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:42.658619 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe927aa8-4814-4fb3-a707-95376a9dd388 admin admin] PATH_INFO: `/v3/domains` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:42.659788 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe927aa8-4814-4fb3-a707-95376a9dd388 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 12:02:42.660358 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe927aa8-4814-4fb3-a707-95376a9dd388 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:02:42.663547 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe927aa8-4814-4fb3-a707-95376a9dd388 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:02:42.687265 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1035/2064] 10.222.0.59 () {58 vars in 1153 bytes} [Tue Oct 11 12:02:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 12:02:42.692537 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8319ebb5-be91-4d16-b3f7-2abd71e917ee None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:02:42.707124 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8319ebb5-be91-4d16-b3f7-2abd71e917ee None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:02:42.714239 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c7b237-1db4-4191-9df3-80e57f8f6789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:02:42.715288 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c7b237-1db4-4191-9df3-80e57f8f6789 admin admin] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:42.715868 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c7b237-1db4-4191-9df3-80e57f8f6789 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:42.716481 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c7b237-1db4-4191-9df3-80e57f8f6789 admin admin] PATH_INFO: `/v3/projects` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:42.717707 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c7b237-1db4-4191-9df3-80e57f8f6789 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:02:42.720685 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c7b237-1db4-4191-9df3-80e57f8f6789 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:02:42.760196 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1030/2065] 10.222.0.59 () {60 vars in 1153 bytes} [Tue Oct 11 12:02:42 2022] POST /identity/v3/projects => generated 389 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:02:42.764724 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe927aa8-4814-4fb3-a707-95376a9dd388 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:02:42.776373 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe927aa8-4814-4fb3-a707-95376a9dd388 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:02:42.868960 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6166a4bd-453a-4252-9cef-7c3f87fa3d7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:02:42.869934 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6166a4bd-453a-4252-9cef-7c3f87fa3d7e admin admin] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:42.870278 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6166a4bd-453a-4252-9cef-7c3f87fa3d7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:42.870621 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6166a4bd-453a-4252-9cef-7c3f87fa3d7e admin admin] PATH_INFO: `/v3/users` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:42.871426 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6166a4bd-453a-4252-9cef-7c3f87fa3d7e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:02:42.874739 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6166a4bd-453a-4252-9cef-7c3f87fa3d7e admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:02:42.907830 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1036/2066] 10.222.0.59 () {60 vars in 1144 bytes} [Tue Oct 11 12:02:42 2022] POST /identity/v3/users => generated 295 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:02:42.912002 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c7b237-1db4-4191-9df3-80e57f8f6789 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:02:42.923724 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c7b237-1db4-4191-9df3-80e57f8f6789 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:02:42.929634 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbc6451-d714-4b06-b08e-c8f32f8e33af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:02:42.930338 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbc6451-d714-4b06-b08e-c8f32f8e33af admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:42.930671 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbc6451-d714-4b06-b08e-c8f32f8e33af admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:42.930946 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbc6451-d714-4b06-b08e-c8f32f8e33af admin admin] PATH_INFO: `/v3/roles` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:42.931611 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbc6451-d714-4b06-b08e-c8f32f8e33af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:02:42.933448 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbc6451-d714-4b06-b08e-c8f32f8e33af admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:02:42.945395 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1031/2067] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:02:42 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 11 12:02:42.950917 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6166a4bd-453a-4252-9cef-7c3f87fa3d7e admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:02:42.969437 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6166a4bd-453a-4252-9cef-7c3f87fa3d7e admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:02:42.978066 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ceff29-671b-4cc7-9e2e-a60118acf67d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:02:42.979059 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ceff29-671b-4cc7-9e2e-a60118acf67d admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:42.979400 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ceff29-671b-4cc7-9e2e-a60118acf67d admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:42.979773 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ceff29-671b-4cc7-9e2e-a60118acf67d admin admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:42.980654 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ceff29-671b-4cc7-9e2e-a60118acf67d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:02:42.982618 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ceff29-671b-4cc7-9e2e-a60118acf67d admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:02:43.018483 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1037/2068] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:02:42 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:02:43.022347 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbc6451-d714-4b06-b08e-c8f32f8e33af admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:02:43.035336 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbc6451-d714-4b06-b08e-c8f32f8e33af admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:02:43.042312 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07b81eb-58c3-4c65-9322-1a392d0bba4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:02:43.043452 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07b81eb-58c3-4c65-9322-1a392d0bba4c admin admin] REQUEST_METHOD: `PUT` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:43.044051 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07b81eb-58c3-4c65-9322-1a392d0bba4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:43.044658 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07b81eb-58c3-4c65-9322-1a392d0bba4c admin admin] PATH_INFO: `/v3/projects/a1e9b761e6df4076b7c385104d7899ad/users/33a7c858b5e44fca8c386b8982c3a95f/roles/de2d2bd26fff4748a79990f97a107164` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:43.068100 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07b81eb-58c3-4c65-9322-1a392d0bba4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1e9b761e6df4076b7c385104d7899ad, user_id=33a7c858b5e44fca8c386b8982c3a95f, role_id=de2d2bd26fff4748a79990f97a107164)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:02:43.069215 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07b81eb-58c3-4c65-9322-1a392d0bba4c admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:02:43.102431 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1032/2069] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 12:02:43 2022] PUT /identity/v3/projects/a1e9b761e6df4076b7c385104d7899ad/users/33a7c858b5e44fca8c386b8982c3a95f/roles/de2d2bd26fff4748a79990f97a107164 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:02:43.114005 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f75470-4a97-4b11-9f50-a349c212a20a None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:43.114385 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f75470-4a97-4b11-9f50-a349c212a20a None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:43.114987 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f75470-4a97-4b11-9f50-a349c212a20a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:43.155846 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-c2f75470-4a97-4b11-9f50-a349c212a20a None None] MFA Rules not processed for user `33a7c858b5e44fca8c386b8982c3a95f`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:02:43.157339 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-c2f75470-4a97-4b11-9f50-a349c212a20a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:02:43.253836 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1038/2070] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:02:43 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:02:43.255423 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07b81eb-58c3-4c65-9322-1a392d0bba4c admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:02:43.348571 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07b81eb-58c3-4c65-9322-1a392d0bba4c admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:02:43.354974 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6554a107-a6d7-4734-bab6-b207a3746e29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:02:43.355903 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6554a107-a6d7-4734-bab6-b207a3746e29 admin admin] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:43.356277 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6554a107-a6d7-4734-bab6-b207a3746e29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:43.356645 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6554a107-a6d7-4734-bab6-b207a3746e29 admin admin] PATH_INFO: `/v3/projects` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:43.357613 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6554a107-a6d7-4734-bab6-b207a3746e29 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:02:43.359670 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6554a107-a6d7-4734-bab6-b207a3746e29 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:02:43.389759 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1033/2071] 10.222.0.59 () {60 vars in 1153 bytes} [Tue Oct 11 12:02:43 2022] POST /identity/v3/projects => generated 389 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 12:02:43.394752 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f75470-4a97-4b11-9f50-a349c212a20a None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:02:43.406416 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f75470-4a97-4b11-9f50-a349c212a20a None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:02:43.420897 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:02:43.479394 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea114aab-8ba2-4a92-bebc-da9743182851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:02:43.480483 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea114aab-8ba2-4a92-bebc-da9743182851 admin admin] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:43.480970 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea114aab-8ba2-4a92-bebc-da9743182851 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:43.481470 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea114aab-8ba2-4a92-bebc-da9743182851 admin admin] PATH_INFO: `/v3/users` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:43.482870 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea114aab-8ba2-4a92-bebc-da9743182851 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:02:43.486850 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea114aab-8ba2-4a92-bebc-da9743182851 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:02:43.516896 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1039/2072] 10.222.0.59 () {60 vars in 1144 bytes} [Tue Oct 11 12:02:43 2022] POST /identity/v3/users => generated 301 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 12:02:43.518673 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6554a107-a6d7-4734-bab6-b207a3746e29 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:02:43.531191 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6554a107-a6d7-4734-bab6-b207a3746e29 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:02:43.536237 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02aee8bf-3b7f-40c9-91e8-d6fa9f03b431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:02:43.536903 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aee8bf-3b7f-40c9-91e8-d6fa9f03b431 admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:43.537170 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aee8bf-3b7f-40c9-91e8-d6fa9f03b431 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:43.537441 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aee8bf-3b7f-40c9-91e8-d6fa9f03b431 admin admin] PATH_INFO: `/v3/roles` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:43.538105 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02aee8bf-3b7f-40c9-91e8-d6fa9f03b431 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:02:43.539780 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02aee8bf-3b7f-40c9-91e8-d6fa9f03b431 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:02:43.552000 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1034/2073] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:02:43 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 11 12:02:43.556659 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea114aab-8ba2-4a92-bebc-da9743182851 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:02:43.569158 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea114aab-8ba2-4a92-bebc-da9743182851 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:02:43.574791 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2cea42-802e-41d2-b012-96056b4e754d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:02:43.575835 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2cea42-802e-41d2-b012-96056b4e754d admin admin] REQUEST_METHOD: `PUT` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:43.576274 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2cea42-802e-41d2-b012-96056b4e754d admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:43.576915 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2cea42-802e-41d2-b012-96056b4e754d admin admin] PATH_INFO: `/v3/projects/c436316b26014358acb871fa07c74411/users/a17c3f5ce34e4d3d9011a9b55b333e73/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:43.604617 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2cea42-802e-41d2-b012-96056b4e754d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c436316b26014358acb871fa07c74411, user_id=a17c3f5ce34e4d3d9011a9b55b333e73, role_id=5cf1f4bd2fef46ab9c6eff56d52f9f69)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:02:43.610027 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2cea42-802e-41d2-b012-96056b4e754d admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:02:43.628577 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1040/2074] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 12:02:43 2022] PUT /identity/v3/projects/c436316b26014358acb871fa07c74411/users/a17c3f5ce34e4d3d9011a9b55b333e73/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:02:43.635376 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02aee8bf-3b7f-40c9-91e8-d6fa9f03b431 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:02:43.646309 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02aee8bf-3b7f-40c9-91e8-d6fa9f03b431 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:02:43.716739 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fb4f46-1a4c-4b2d-8f6f-a194b10c00c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:02:43.717549 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fb4f46-1a4c-4b2d-8f6f-a194b10c00c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:43.718002 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fb4f46-1a4c-4b2d-8f6f-a194b10c00c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:43.718341 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fb4f46-1a4c-4b2d-8f6f-a194b10c00c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:43.719460 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fb4f46-1a4c-4b2d-8f6f-a194b10c00c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:02:43.722385 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fb4f46-1a4c-4b2d-8f6f-a194b10c00c6 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:02:43.736581 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1035/2075] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:02:43 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:02:43.741350 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2cea42-802e-41d2-b012-96056b4e754d admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:02:43.759247 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2cea42-802e-41d2-b012-96056b4e754d admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:02:43.767875 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412ab52d-5489-4d9f-925b-fe3f177cc6b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:02:43.769119 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412ab52d-5489-4d9f-925b-fe3f177cc6b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:43.769408 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412ab52d-5489-4d9f-925b-fe3f177cc6b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:43.773897 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412ab52d-5489-4d9f-925b-fe3f177cc6b1 admin admin] PATH_INFO: `/v3/projects/c436316b26014358acb871fa07c74411/users/a17c3f5ce34e4d3d9011a9b55b333e73/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:43.777379 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412ab52d-5489-4d9f-925b-fe3f177cc6b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c436316b26014358acb871fa07c74411, user_id=a17c3f5ce34e4d3d9011a9b55b333e73, role_id=5cf1f4bd2fef46ab9c6eff56d52f9f69)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:02:43.783079 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412ab52d-5489-4d9f-925b-fe3f177cc6b1 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:02:43.802486 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1041/2076] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 12:02:43 2022] PUT /identity/v3/projects/c436316b26014358acb871fa07c74411/users/a17c3f5ce34e4d3d9011a9b55b333e73/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:02:43.817142 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14117649-cda8-45e9-8fdb-1d3ccad78999 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:43.817474 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14117649-cda8-45e9-8fdb-1d3ccad78999 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:43.817706 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14117649-cda8-45e9-8fdb-1d3ccad78999 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:43.827729 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:02:43.859750 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-14117649-cda8-45e9-8fdb-1d3ccad78999 None None] MFA Rules not processed for user `a17c3f5ce34e4d3d9011a9b55b333e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:02:43.860374 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-14117649-cda8-45e9-8fdb-1d3ccad78999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:02:43.957000 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1036/2077] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:02:43 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 12:02:43.970003 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412ab52d-5489-4d9f-925b-fe3f177cc6b1 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:02:43.973243 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-412ab52d-5489-4d9f-925b-fe3f177cc6b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:02:44.071773 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412ab52d-5489-4d9f-925b-fe3f177cc6b1 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:02:44.077132 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f4162b-c4e6-434f-b099-83d6837d7d17 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b1b651c73664d08accc6edf2f79bace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:02:44.078504 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f4162b-c4e6-434f-b099-83d6837d7d17 service cinder] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:44.078729 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f4162b-c4e6-434f-b099-83d6837d7d17 service cinder] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:44.079196 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f4162b-c4e6-434f-b099-83d6837d7d17 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:44.093711 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f4162b-c4e6-434f-b099-83d6837d7d17 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:02:44.095391 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f4162b-c4e6-434f-b099-83d6837d7d17 service cinder] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:02:44.108222 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1042/2078] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:02:43 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:02:44.265413 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14117649-cda8-45e9-8fdb-1d3ccad78999 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:02:44.343959 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14117649-cda8-45e9-8fdb-1d3ccad78999 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:02:44.348767 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea8e6d2-59aa-4997-a1d8-935201ec1c71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:02:44.349420 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea8e6d2-59aa-4997-a1d8-935201ec1c71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:44.349718 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea8e6d2-59aa-4997-a1d8-935201ec1c71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:44.349999 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea8e6d2-59aa-4997-a1d8-935201ec1c71 admin admin] PATH_INFO: `/v3/users/33a7c858b5e44fca8c386b8982c3a95f` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:44.351283 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea8e6d2-59aa-4997-a1d8-935201ec1c71 admin admin] RBAC: Authorizing `identity:delete_user(user_id=33a7c858b5e44fca8c386b8982c3a95f)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:02:44.353230 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea8e6d2-59aa-4997-a1d8-935201ec1c71 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:02:44.426845 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-8ea8e6d2-59aa-4997-a1d8-935201ec1c71 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33a7c858b5e44fca8c386b8982c3a95f'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:44.439586 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-8ea8e6d2-59aa-4997-a1d8-935201ec1c71 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33a7c858b5e44fca8c386b8982c3a95f'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:44.449345 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-8ea8e6d2-59aa-4997-a1d8-935201ec1c71 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33a7c858b5e44fca8c386b8982c3a95f'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:44.450106 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-8ea8e6d2-59aa-4997-a1d8-935201ec1c71 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33a7c858b5e44fca8c386b8982c3a95f'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:44.457749 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-8ea8e6d2-59aa-4997-a1d8-935201ec1c71 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33a7c858b5e44fca8c386b8982c3a95f'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:44.463818 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1037/2079] 10.222.0.59 () {58 vars in 1224 bytes} [Tue Oct 11 12:02:44 2022] DELETE /identity/v3/users/33a7c858b5e44fca8c386b8982c3a95f => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:02:44.468516 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd84a36-212e-4f56-ac18-72a993ff478c None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:44.468867 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd84a36-212e-4f56-ac18-72a993ff478c None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:44.469234 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd84a36-212e-4f56-ac18-72a993ff478c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:44.488995 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-ebd84a36-212e-4f56-ac18-72a993ff478c None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:02:44.490322 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-ebd84a36-212e-4f56-ac18-72a993ff478c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:02:44.558457 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1043/2080] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:02:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:02:44.568655 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea8e6d2-59aa-4997-a1d8-935201ec1c71 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:02:44.570191 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-8ea8e6d2-59aa-4997-a1d8-935201ec1c71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:02:44.584137 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:02:44.669630 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea8e6d2-59aa-4997-a1d8-935201ec1c71 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:02:44.673725 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbcdb78-2e10-4ba4-9ecd-4243d7f626b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e563fe174db4cc19ae6bf8162cc3d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:02:44.674354 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbcdb78-2e10-4ba4-9ecd-4243d7f626b5 service neutron] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:44.674656 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbcdb78-2e10-4ba4-9ecd-4243d7f626b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:44.674928 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbcdb78-2e10-4ba4-9ecd-4243d7f626b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:44.686221 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbcdb78-2e10-4ba4-9ecd-4243d7f626b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:02:44.687797 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbcdb78-2e10-4ba4-9ecd-4243d7f626b5 service neutron] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:02:44.699579 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1038/2081] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:02:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:02:44.984363 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd84a36-212e-4f56-ac18-72a993ff478c None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:02:44.987476 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-ebd84a36-212e-4f56-ac18-72a993ff478c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:02:45.007642 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd84a36-212e-4f56-ac18-72a993ff478c None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:02:45.013857 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad0cf9e-de72-469a-beb6-611c73f2d4c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:02:45.014695 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad0cf9e-de72-469a-beb6-611c73f2d4c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:45.015060 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad0cf9e-de72-469a-beb6-611c73f2d4c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:45.015435 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad0cf9e-de72-469a-beb6-611c73f2d4c3 admin admin] PATH_INFO: `/v3/projects/a1e9b761e6df4076b7c385104d7899ad` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:45.017695 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad0cf9e-de72-469a-beb6-611c73f2d4c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1e9b761e6df4076b7c385104d7899ad)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:02:45.021476 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad0cf9e-de72-469a-beb6-611c73f2d4c3 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:02:45.036853 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:02:45.091578 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-7ad0cf9e-de72-469a-beb6-611c73f2d4c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1e9b761e6df4076b7c385104d7899ad', 'request_id': 'req-7ad0cf9e-de72-469a-beb6-611c73f2d4c3'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:45.098513 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-7ad0cf9e-de72-469a-beb6-611c73f2d4c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1e9b761e6df4076b7c385104d7899ad', 'request_id': 'req-7ad0cf9e-de72-469a-beb6-611c73f2d4c3'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:45.106222 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-7ad0cf9e-de72-469a-beb6-611c73f2d4c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1e9b761e6df4076b7c385104d7899ad', 'request_id': 'req-7ad0cf9e-de72-469a-beb6-611c73f2d4c3'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:45.129668 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-7ad0cf9e-de72-469a-beb6-611c73f2d4c3 admin admin] The token cache is being invalidate because project a1e9b761e6df4076b7c385104d7899ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81432) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 12:02:45.129964 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-7ad0cf9e-de72-469a-beb6-611c73f2d4c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:45.130834 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-7ad0cf9e-de72-469a-beb6-611c73f2d4c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:45.132667 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1044/2082] 10.222.0.59 () {58 vars in 1233 bytes} [Tue Oct 11 12:02:44 2022] DELETE /identity/v3/projects/a1e9b761e6df4076b7c385104d7899ad => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:02:45.136830 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbcdb78-2e10-4ba4-9ecd-4243d7f626b5 service neutron] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:02:45.147662 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbcdb78-2e10-4ba4-9ecd-4243d7f626b5 service neutron] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:02:45.217842 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf876a8-7b1a-41e2-a9dd-79b409e58c76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:02:45.218502 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf876a8-7b1a-41e2-a9dd-79b409e58c76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:45.218847 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf876a8-7b1a-41e2-a9dd-79b409e58c76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:45.219155 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf876a8-7b1a-41e2-a9dd-79b409e58c76 admin admin] PATH_INFO: `/v3/users/a17c3f5ce34e4d3d9011a9b55b333e73` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:45.220632 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf876a8-7b1a-41e2-a9dd-79b409e58c76 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a17c3f5ce34e4d3d9011a9b55b333e73)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:02:45.222419 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf876a8-7b1a-41e2-a9dd-79b409e58c76 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:02:45.280190 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-0bf876a8-7b1a-41e2-a9dd-79b409e58c76 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a17c3f5ce34e4d3d9011a9b55b333e73'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:45.295093 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-0bf876a8-7b1a-41e2-a9dd-79b409e58c76 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a17c3f5ce34e4d3d9011a9b55b333e73'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:45.303901 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-0bf876a8-7b1a-41e2-a9dd-79b409e58c76 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a17c3f5ce34e4d3d9011a9b55b333e73'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:45.304564 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-0bf876a8-7b1a-41e2-a9dd-79b409e58c76 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a17c3f5ce34e4d3d9011a9b55b333e73'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:45.313516 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-0bf876a8-7b1a-41e2-a9dd-79b409e58c76 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a17c3f5ce34e4d3d9011a9b55b333e73'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:45.321659 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1039/2083] 10.222.0.59 () {58 vars in 1224 bytes} [Tue Oct 11 12:02:45 2022] DELETE /identity/v3/users/a17c3f5ce34e4d3d9011a9b55b333e73 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:02:45.712221 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad0cf9e-de72-469a-beb6-611c73f2d4c3 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:02:45.714750 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-7ad0cf9e-de72-469a-beb6-611c73f2d4c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:02:45.737568 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad0cf9e-de72-469a-beb6-611c73f2d4c3 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:02:45.810493 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3847f578-ea74-4735-9243-b650a7054138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:02:45.811245 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3847f578-ea74-4735-9243-b650a7054138 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:45.811574 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3847f578-ea74-4735-9243-b650a7054138 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:45.811915 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3847f578-ea74-4735-9243-b650a7054138 admin admin] PATH_INFO: `/v3/projects/c436316b26014358acb871fa07c74411` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:45.813338 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3847f578-ea74-4735-9243-b650a7054138 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c436316b26014358acb871fa07c74411)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:02:45.815484 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3847f578-ea74-4735-9243-b650a7054138 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:02:45.899312 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-3847f578-ea74-4735-9243-b650a7054138 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c436316b26014358acb871fa07c74411', 'request_id': 'req-3847f578-ea74-4735-9243-b650a7054138'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:45.911526 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-3847f578-ea74-4735-9243-b650a7054138 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c436316b26014358acb871fa07c74411', 'request_id': 'req-3847f578-ea74-4735-9243-b650a7054138'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:45.920187 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-3847f578-ea74-4735-9243-b650a7054138 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c436316b26014358acb871fa07c74411', 'request_id': 'req-3847f578-ea74-4735-9243-b650a7054138'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:45.947151 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-3847f578-ea74-4735-9243-b650a7054138 admin admin] The token cache is being invalidate because project c436316b26014358acb871fa07c74411 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81432) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 12:02:45.947671 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-3847f578-ea74-4735-9243-b650a7054138 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:45.948771 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-3847f578-ea74-4735-9243-b650a7054138 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:45.951535 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1045/2084] 10.222.0.59 () {58 vars in 1233 bytes} [Tue Oct 11 12:02:45 2022] DELETE /identity/v3/projects/c436316b26014358acb871fa07c74411 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:02:45.959707 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b86fd3e-3810-4e76-ba63-636d377cc244 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:45.960197 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b86fd3e-3810-4e76-ba63-636d377cc244 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:45.960706 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b86fd3e-3810-4e76-ba63-636d377cc244 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:45.994194 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-3b86fd3e-3810-4e76-ba63-636d377cc244 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:02:45.996423 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-3b86fd3e-3810-4e76-ba63-636d377cc244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:02:46.045522 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:02:46.100119 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1040/2085] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:02:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 12:02:46.110030 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03d8bfa-d5d9-42f2-94f2-264d6e7df812 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:46.110432 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03d8bfa-d5d9-42f2-94f2-264d6e7df812 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:46.110824 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03d8bfa-d5d9-42f2-94f2-264d6e7df812 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:46.134727 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-e03d8bfa-d5d9-42f2-94f2-264d6e7df812 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:02:46.135968 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-e03d8bfa-d5d9-42f2-94f2-264d6e7df812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:02:46.144495 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1046/2086] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:02:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:02:46.149306 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b86fd3e-3810-4e76-ba63-636d377cc244 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:02:46.160605 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b86fd3e-3810-4e76-ba63-636d377cc244 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:02:46.165567 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd42e1d1-30b8-4482-904a-6216f460ad3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:02:46.166198 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd42e1d1-30b8-4482-904a-6216f460ad3b admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:46.166510 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd42e1d1-30b8-4482-904a-6216f460ad3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:46.166822 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd42e1d1-30b8-4482-904a-6216f460ad3b admin admin] PATH_INFO: `/v3/domains` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:46.167486 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd42e1d1-30b8-4482-904a-6216f460ad3b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 12:02:46.167789 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd42e1d1-30b8-4482-904a-6216f460ad3b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:02:46.169414 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd42e1d1-30b8-4482-904a-6216f460ad3b admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:02:46.182327 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1041/2087] 10.222.0.59 () {58 vars in 1153 bytes} [Tue Oct 11 12:02:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 12:02:46.186972 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03d8bfa-d5d9-42f2-94f2-264d6e7df812 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:02:46.196610 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03d8bfa-d5d9-42f2-94f2-264d6e7df812 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:02:46.201528 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c275c97-a0d8-4bc1-9fa2-ddb930641d90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:02:46.202174 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c275c97-a0d8-4bc1-9fa2-ddb930641d90 admin admin] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:46.202451 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c275c97-a0d8-4bc1-9fa2-ddb930641d90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:46.202755 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c275c97-a0d8-4bc1-9fa2-ddb930641d90 admin admin] PATH_INFO: `/v3/projects` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:46.203398 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c275c97-a0d8-4bc1-9fa2-ddb930641d90 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:02:46.205321 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c275c97-a0d8-4bc1-9fa2-ddb930641d90 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:02:46.231121 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:02:46.241125 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1047/2088] 10.222.0.59 () {60 vars in 1153 bytes} [Tue Oct 11 12:02:46 2022] POST /identity/v3/projects => generated 383 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:02:46.245459 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd42e1d1-30b8-4482-904a-6216f460ad3b admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:02:46.257819 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd42e1d1-30b8-4482-904a-6216f460ad3b admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:02:46.352041 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f948340b-2a4f-4169-8b87-d03c4982e580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:02:46.353976 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f948340b-2a4f-4169-8b87-d03c4982e580 admin admin] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:46.353976 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f948340b-2a4f-4169-8b87-d03c4982e580 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:46.353976 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f948340b-2a4f-4169-8b87-d03c4982e580 admin admin] PATH_INFO: `/v3/users` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:46.353976 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f948340b-2a4f-4169-8b87-d03c4982e580 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:02:46.355874 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f948340b-2a4f-4169-8b87-d03c4982e580 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:02:46.384518 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1042/2089] 10.222.0.59 () {60 vars in 1144 bytes} [Tue Oct 11 12:02:46 2022] POST /identity/v3/users => generated 292 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:02:46.385801 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c275c97-a0d8-4bc1-9fa2-ddb930641d90 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:02:46.396537 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c275c97-a0d8-4bc1-9fa2-ddb930641d90 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:02:46.402719 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b9cd7a-7b90-4f8a-8157-27dc4581b64e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:02:46.405502 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b9cd7a-7b90-4f8a-8157-27dc4581b64e admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:46.405502 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b9cd7a-7b90-4f8a-8157-27dc4581b64e admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:46.405502 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b9cd7a-7b90-4f8a-8157-27dc4581b64e admin admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:46.405502 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b9cd7a-7b90-4f8a-8157-27dc4581b64e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:02:46.406191 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b9cd7a-7b90-4f8a-8157-27dc4581b64e admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:02:46.440620 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1048/2090] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:02:46 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 11 12:02:46.444475 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f948340b-2a4f-4169-8b87-d03c4982e580 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:02:46.458163 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f948340b-2a4f-4169-8b87-d03c4982e580 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:02:46.463193 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4c4d94-e61a-4134-81de-89b83b97384a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:02:46.464068 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4c4d94-e61a-4134-81de-89b83b97384a admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:46.464361 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4c4d94-e61a-4134-81de-89b83b97384a admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:46.464635 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4c4d94-e61a-4134-81de-89b83b97384a admin admin] PATH_INFO: `/v3/roles` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:46.465882 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4c4d94-e61a-4134-81de-89b83b97384a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:02:46.467030 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4c4d94-e61a-4134-81de-89b83b97384a admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:02:46.482893 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1043/2091] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:02:46 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:02:46.487475 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b9cd7a-7b90-4f8a-8157-27dc4581b64e admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:02:46.507338 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b9cd7a-7b90-4f8a-8157-27dc4581b64e admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:02:46.514845 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b525d8c2-4520-403e-b7f2-a246e4643569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:02:46.516390 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b525d8c2-4520-403e-b7f2-a246e4643569 admin admin] REQUEST_METHOD: `PUT` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:46.517572 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b525d8c2-4520-403e-b7f2-a246e4643569 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:46.518055 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b525d8c2-4520-403e-b7f2-a246e4643569 admin admin] PATH_INFO: `/v3/projects/d6841a68b8274b5da6c8ac64035ac7b4/users/5e0688bd9f884907b7eefe56c8da8b16/roles/de2d2bd26fff4748a79990f97a107164` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:46.552431 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b525d8c2-4520-403e-b7f2-a246e4643569 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6841a68b8274b5da6c8ac64035ac7b4, user_id=5e0688bd9f884907b7eefe56c8da8b16, role_id=de2d2bd26fff4748a79990f97a107164)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:02:46.565219 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b525d8c2-4520-403e-b7f2-a246e4643569 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:02:46.594618 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1049/2092] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 12:02:46 2022] PUT /identity/v3/projects/d6841a68b8274b5da6c8ac64035ac7b4/users/5e0688bd9f884907b7eefe56c8da8b16/roles/de2d2bd26fff4748a79990f97a107164 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 12:02:46.672276 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf87e14-694f-41c4-ae9d-f600fd95d9e3 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:46.672564 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf87e14-694f-41c4-ae9d-f600fd95d9e3 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:46.672842 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf87e14-694f-41c4-ae9d-f600fd95d9e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:46.710229 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-0bf87e14-694f-41c4-ae9d-f600fd95d9e3 None None] MFA Rules not processed for user `5e0688bd9f884907b7eefe56c8da8b16`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:02:46.711570 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-0bf87e14-694f-41c4-ae9d-f600fd95d9e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:02:46.833190 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1044/2093] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:02:46 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:02:46.834341 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b525d8c2-4520-403e-b7f2-a246e4643569 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:02:46.949920 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b525d8c2-4520-403e-b7f2-a246e4643569 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:02:46.955572 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a293cb7a-63d7-45dc-b3f6-0ad6103b1745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:02:46.956301 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a293cb7a-63d7-45dc-b3f6-0ad6103b1745 admin admin] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:46.956644 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a293cb7a-63d7-45dc-b3f6-0ad6103b1745 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:46.956983 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a293cb7a-63d7-45dc-b3f6-0ad6103b1745 admin admin] PATH_INFO: `/v3/projects` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:46.957917 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a293cb7a-63d7-45dc-b3f6-0ad6103b1745 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:02:46.964058 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a293cb7a-63d7-45dc-b3f6-0ad6103b1745 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:02:46.991944 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1050/2094] 10.222.0.59 () {60 vars in 1153 bytes} [Tue Oct 11 12:02:46 2022] POST /identity/v3/projects => generated 383 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 12:02:46.996704 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf87e14-694f-41c4-ae9d-f600fd95d9e3 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:02:47.008181 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf87e14-694f-41c4-ae9d-f600fd95d9e3 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:02:47.036097 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:02:47.072866 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb4043a-765b-4d8f-96ab-7c4ad19a04c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:02:47.073320 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb4043a-765b-4d8f-96ab-7c4ad19a04c2 admin admin] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:47.073624 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb4043a-765b-4d8f-96ab-7c4ad19a04c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:47.073903 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb4043a-765b-4d8f-96ab-7c4ad19a04c2 admin admin] PATH_INFO: `/v3/users` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:47.074641 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb4043a-765b-4d8f-96ab-7c4ad19a04c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:02:47.076632 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb4043a-765b-4d8f-96ab-7c4ad19a04c2 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:02:47.109587 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1045/2095] 10.222.0.59 () {60 vars in 1144 bytes} [Tue Oct 11 12:02:46 2022] POST /identity/v3/users => generated 292 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:02:47.113748 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a293cb7a-63d7-45dc-b3f6-0ad6103b1745 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:02:47.127248 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a293cb7a-63d7-45dc-b3f6-0ad6103b1745 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:02:47.132335 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23391a38-8f37-4899-9453-5ebac46c04ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:02:47.133043 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23391a38-8f37-4899-9453-5ebac46c04ef admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:47.133323 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23391a38-8f37-4899-9453-5ebac46c04ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:47.133624 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23391a38-8f37-4899-9453-5ebac46c04ef admin admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:47.134380 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23391a38-8f37-4899-9453-5ebac46c04ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:02:47.136053 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23391a38-8f37-4899-9453-5ebac46c04ef admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:02:47.148512 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1051/2096] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:02:47 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:02:47.153367 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb4043a-765b-4d8f-96ab-7c4ad19a04c2 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:02:47.164133 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb4043a-765b-4d8f-96ab-7c4ad19a04c2 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:02:47.169406 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528964b7-d50d-46d1-8bff-5ad440d2a870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:02:47.170087 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528964b7-d50d-46d1-8bff-5ad440d2a870 admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:47.170472 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528964b7-d50d-46d1-8bff-5ad440d2a870 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:47.180386 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528964b7-d50d-46d1-8bff-5ad440d2a870 admin admin] PATH_INFO: `/v3/roles` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:47.180386 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528964b7-d50d-46d1-8bff-5ad440d2a870 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:02:47.180386 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528964b7-d50d-46d1-8bff-5ad440d2a870 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:02:47.185623 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1046/2097] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:02:47 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:02:47.190147 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23391a38-8f37-4899-9453-5ebac46c04ef admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:02:47.201295 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23391a38-8f37-4899-9453-5ebac46c04ef admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:02:47.208423 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa7f291-ebe5-4107-a8ca-9bf716120701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:02:47.209110 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa7f291-ebe5-4107-a8ca-9bf716120701 admin admin] REQUEST_METHOD: `PUT` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:47.209387 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa7f291-ebe5-4107-a8ca-9bf716120701 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:47.209661 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa7f291-ebe5-4107-a8ca-9bf716120701 admin admin] PATH_INFO: `/v3/projects/86aad02683e44551a8d83923121b56ca/users/a35edc63c4d34a7ba5f0c160eb5e6516/roles/de2d2bd26fff4748a79990f97a107164` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:47.226400 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa7f291-ebe5-4107-a8ca-9bf716120701 admin admin] RBAC: Authorizing `identity:create_grant(project_id=86aad02683e44551a8d83923121b56ca, user_id=a35edc63c4d34a7ba5f0c160eb5e6516, role_id=de2d2bd26fff4748a79990f97a107164)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:02:47.229046 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa7f291-ebe5-4107-a8ca-9bf716120701 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:02:47.245029 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1052/2098] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 12:02:47 2022] PUT /identity/v3/projects/86aad02683e44551a8d83923121b56ca/users/a35edc63c4d34a7ba5f0c160eb5e6516/roles/de2d2bd26fff4748a79990f97a107164 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:02:47.255972 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f237181-2a39-4fda-871c-49f81327dc04 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:47.256279 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f237181-2a39-4fda-871c-49f81327dc04 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:47.256717 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f237181-2a39-4fda-871c-49f81327dc04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:47.288732 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-9f237181-2a39-4fda-871c-49f81327dc04 None None] MFA Rules not processed for user `a35edc63c4d34a7ba5f0c160eb5e6516`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:02:47.289988 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-9f237181-2a39-4fda-871c-49f81327dc04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:02:47.366196 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1047/2099] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:02:47 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 12:02:47.370906 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa7f291-ebe5-4107-a8ca-9bf716120701 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:02:47.438713 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa7f291-ebe5-4107-a8ca-9bf716120701 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:02:47.443256 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719dc8b6-72a6-4b61-8d15-eaf4f94ca0eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:02:47.443900 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719dc8b6-72a6-4b61-8d15-eaf4f94ca0eb admin admin] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:47.444165 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719dc8b6-72a6-4b61-8d15-eaf4f94ca0eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:47.444434 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719dc8b6-72a6-4b61-8d15-eaf4f94ca0eb admin admin] PATH_INFO: `/v3/projects` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:47.445157 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719dc8b6-72a6-4b61-8d15-eaf4f94ca0eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:02:47.447214 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719dc8b6-72a6-4b61-8d15-eaf4f94ca0eb admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:02:47.473746 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:02:47.492239 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1053/2100] 10.222.0.59 () {60 vars in 1153 bytes} [Tue Oct 11 12:02:47 2022] POST /identity/v3/projects => generated 385 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 12:02:47.493639 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f237181-2a39-4fda-871c-49f81327dc04 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:02:47.506378 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f237181-2a39-4fda-871c-49f81327dc04 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:02:47.567301 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc865391-e9af-42d6-86e3-2ce584c5a9a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:02:47.568319 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc865391-e9af-42d6-86e3-2ce584c5a9a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:47.568319 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc865391-e9af-42d6-86e3-2ce584c5a9a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:47.568555 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc865391-e9af-42d6-86e3-2ce584c5a9a6 admin admin] PATH_INFO: `/v3/users` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:47.569119 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc865391-e9af-42d6-86e3-2ce584c5a9a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:02:47.571129 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc865391-e9af-42d6-86e3-2ce584c5a9a6 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:02:47.605315 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1048/2101] 10.222.0.59 () {60 vars in 1144 bytes} [Tue Oct 11 12:02:47 2022] POST /identity/v3/users => generated 299 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:02:47.613167 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719dc8b6-72a6-4b61-8d15-eaf4f94ca0eb admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:02:47.627523 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719dc8b6-72a6-4b61-8d15-eaf4f94ca0eb admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:02:47.633977 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642abdd0-9cf6-4d28-821f-d02bde2f527b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:02:47.634949 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642abdd0-9cf6-4d28-821f-d02bde2f527b admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:47.635477 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642abdd0-9cf6-4d28-821f-d02bde2f527b admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:47.636039 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642abdd0-9cf6-4d28-821f-d02bde2f527b admin admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:47.637058 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642abdd0-9cf6-4d28-821f-d02bde2f527b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:02:47.639194 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642abdd0-9cf6-4d28-821f-d02bde2f527b admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:02:47.658676 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1054/2102] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:02:47 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:02:47.664366 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc865391-e9af-42d6-86e3-2ce584c5a9a6 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:02:47.684565 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc865391-e9af-42d6-86e3-2ce584c5a9a6 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:02:47.692684 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1438f78d-a027-4583-8c84-05e70b4e14ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:02:47.693884 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1438f78d-a027-4583-8c84-05e70b4e14ef admin admin] REQUEST_METHOD: `PUT` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:47.694410 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1438f78d-a027-4583-8c84-05e70b4e14ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:47.695009 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1438f78d-a027-4583-8c84-05e70b4e14ef admin admin] PATH_INFO: `/v3/projects/966b16a80e804c0a9ea2501af937587a/users/752435fcc1474c8f80d5cc9d825f9df3/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:47.723296 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1438f78d-a027-4583-8c84-05e70b4e14ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=966b16a80e804c0a9ea2501af937587a, user_id=752435fcc1474c8f80d5cc9d825f9df3, role_id=5cf1f4bd2fef46ab9c6eff56d52f9f69)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:02:47.729338 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1438f78d-a027-4583-8c84-05e70b4e14ef admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:02:47.751043 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:02:47.767677 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642abdd0-9cf6-4d28-821f-d02bde2f527b admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:02:47.769422 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1049/2103] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 12:02:47 2022] PUT /identity/v3/projects/966b16a80e804c0a9ea2501af937587a/users/752435fcc1474c8f80d5cc9d825f9df3/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:02:47.783519 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642abdd0-9cf6-4d28-821f-d02bde2f527b admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:02:47.896138 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c855096d-69f3-4a2c-9ac9-c91ae0cb897c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:02:47.899003 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c855096d-69f3-4a2c-9ac9-c91ae0cb897c admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:47.899782 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c855096d-69f3-4a2c-9ac9-c91ae0cb897c admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:47.900886 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c855096d-69f3-4a2c-9ac9-c91ae0cb897c admin admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:47.902409 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c855096d-69f3-4a2c-9ac9-c91ae0cb897c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:02:47.906237 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c855096d-69f3-4a2c-9ac9-c91ae0cb897c admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:02:47.948426 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1055/2104] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:02:47 2022] GET /identity/v3/roles => generated 1420 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:02:47.954120 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1438f78d-a027-4583-8c84-05e70b4e14ef admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:02:47.976456 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1438f78d-a027-4583-8c84-05e70b4e14ef admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:02:47.983558 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc3f8ef-36f8-4cd0-9956-d0b6815ed0e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:02:47.984308 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc3f8ef-36f8-4cd0-9956-d0b6815ed0e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:47.984603 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc3f8ef-36f8-4cd0-9956-d0b6815ed0e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:47.984868 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc3f8ef-36f8-4cd0-9956-d0b6815ed0e2 admin admin] PATH_INFO: `/v3/projects/966b16a80e804c0a9ea2501af937587a/users/752435fcc1474c8f80d5cc9d825f9df3/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:47.987025 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc3f8ef-36f8-4cd0-9956-d0b6815ed0e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=966b16a80e804c0a9ea2501af937587a, user_id=752435fcc1474c8f80d5cc9d825f9df3, role_id=5cf1f4bd2fef46ab9c6eff56d52f9f69)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:02:47.989829 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc3f8ef-36f8-4cd0-9956-d0b6815ed0e2 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:02:48.001928 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1050/2105] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 12:02:47 2022] PUT /identity/v3/projects/966b16a80e804c0a9ea2501af937587a/users/752435fcc1474c8f80d5cc9d825f9df3/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:02:48.013551 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eedc018-4fa3-44cb-ab89-77d48d14bd9e None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:48.013946 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eedc018-4fa3-44cb-ab89-77d48d14bd9e None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:48.014285 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eedc018-4fa3-44cb-ab89-77d48d14bd9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:48.048948 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-9eedc018-4fa3-44cb-ab89-77d48d14bd9e None None] MFA Rules not processed for user `752435fcc1474c8f80d5cc9d825f9df3`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:02:48.050295 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-9eedc018-4fa3-44cb-ab89-77d48d14bd9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:02:48.142597 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1056/2106] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:02:48 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:02:48.155725 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc3f8ef-36f8-4cd0-9956-d0b6815ed0e2 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:02:48.158150 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-4cc3f8ef-36f8-4cd0-9956-d0b6815ed0e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:02:48.250006 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc3f8ef-36f8-4cd0-9956-d0b6815ed0e2 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:02:48.254133 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceffd390-a3e4-4b50-b28f-7d1aaa607793 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b1b651c73664d08accc6edf2f79bace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:02:48.254887 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceffd390-a3e4-4b50-b28f-7d1aaa607793 service cinder] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:48.255245 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceffd390-a3e4-4b50-b28f-7d1aaa607793 service cinder] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:48.255575 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceffd390-a3e4-4b50-b28f-7d1aaa607793 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:48.359703 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceffd390-a3e4-4b50-b28f-7d1aaa607793 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:02:48.362535 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceffd390-a3e4-4b50-b28f-7d1aaa607793 service cinder] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:02:48.382224 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1051/2107] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:02:48 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:02:49.351477 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74caa273-2adf-46f6-b882-324a340b4b53 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:49.351995 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74caa273-2adf-46f6-b882-324a340b4b53 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:49.351995 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74caa273-2adf-46f6-b882-324a340b4b53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:49.373186 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-74caa273-2adf-46f6-b882-324a340b4b53 None None] MFA Rules not processed for user `c58107eee1b94541a330c26d7b82a53b`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:02:49.374437 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-74caa273-2adf-46f6-b882-324a340b4b53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:02:49.432479 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1057/2108] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:02:49 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 12:02:50.024824 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceffd390-a3e4-4b50-b28f-7d1aaa607793 service cinder] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:02:50.037750 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceffd390-a3e4-4b50-b28f-7d1aaa607793 service cinder] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:02:50.042079 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470506c2-f741-42d8-9ddd-0f5e32a3215c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b1b651c73664d08accc6edf2f79bace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:02:50.042919 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470506c2-f741-42d8-9ddd-0f5e32a3215c service cinder] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:50.043204 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470506c2-f741-42d8-9ddd-0f5e32a3215c service cinder] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:50.043571 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470506c2-f741-42d8-9ddd-0f5e32a3215c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:50.114638 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470506c2-f741-42d8-9ddd-0f5e32a3215c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:02:50.116578 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470506c2-f741-42d8-9ddd-0f5e32a3215c service cinder] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:02:50.131805 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1052/2109] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:02:50 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:02:50.954137 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74caa273-2adf-46f6-b882-324a340b4b53 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:02:50.973961 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74caa273-2adf-46f6-b882-324a340b4b53 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:02:50.980164 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7009ed86-84e9-4aca-a080-6305347cb3e8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b1b651c73664d08accc6edf2f79bace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:02:50.980877 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7009ed86-84e9-4aca-a080-6305347cb3e8 service cinder] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:50.981177 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7009ed86-84e9-4aca-a080-6305347cb3e8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:50.981453 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7009ed86-84e9-4aca-a080-6305347cb3e8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:50.994442 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7009ed86-84e9-4aca-a080-6305347cb3e8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:02:50.997348 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7009ed86-84e9-4aca-a080-6305347cb3e8 service cinder] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:02:51.007252 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:02:51.017109 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1058/2110] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:02:50 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:02:52.549099 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470506c2-f741-42d8-9ddd-0f5e32a3215c service cinder] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:02:52.551266 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-470506c2-f741-42d8-9ddd-0f5e32a3215c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:02:52.637160 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:02:52.693978 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470506c2-f741-42d8-9ddd-0f5e32a3215c service cinder] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:02:52.700824 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeae56d5-a92c-4430-888a-4291c71d444d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:02:52.701995 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeae56d5-a92c-4430-888a-4291c71d444d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:52.702282 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeae56d5-a92c-4430-888a-4291c71d444d admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:52.702671 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeae56d5-a92c-4430-888a-4291c71d444d admin admin] PATH_INFO: `/v3/users/42c13366d0644388a703beaab38a6ead` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:52.704644 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeae56d5-a92c-4430-888a-4291c71d444d admin admin] RBAC: Authorizing `identity:delete_user(user_id=42c13366d0644388a703beaab38a6ead)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:02:52.706810 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeae56d5-a92c-4430-888a-4291c71d444d admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:02:52.776141 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-eeae56d5-a92c-4430-888a-4291c71d444d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42c13366d0644388a703beaab38a6ead'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:52.804263 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-eeae56d5-a92c-4430-888a-4291c71d444d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42c13366d0644388a703beaab38a6ead'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:52.815723 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-eeae56d5-a92c-4430-888a-4291c71d444d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42c13366d0644388a703beaab38a6ead'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:52.816419 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-eeae56d5-a92c-4430-888a-4291c71d444d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42c13366d0644388a703beaab38a6ead'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:52.827156 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-eeae56d5-a92c-4430-888a-4291c71d444d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42c13366d0644388a703beaab38a6ead'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:52.839400 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1053/2111] 10.222.0.59 () {58 vars in 1224 bytes} [Tue Oct 11 12:02:52 2022] DELETE /identity/v3/users/42c13366d0644388a703beaab38a6ead => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:02:52.841200 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa93da4e-1d87-47b9-8ab5-4cde864a72be None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:52.842031 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa93da4e-1d87-47b9-8ab5-4cde864a72be None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:52.842031 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa93da4e-1d87-47b9-8ab5-4cde864a72be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:52.872700 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-fa93da4e-1d87-47b9-8ab5-4cde864a72be None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:02:52.873991 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-fa93da4e-1d87-47b9-8ab5-4cde864a72be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:02:52.962325 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1059/2112] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:02:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:02:52.978338 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeae56d5-a92c-4430-888a-4291c71d444d admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:02:52.981174 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-eeae56d5-a92c-4430-888a-4291c71d444d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:02:53.069321 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeae56d5-a92c-4430-888a-4291c71d444d admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:02:53.073136 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337a79d7-eeb9-4f73-a8f6-5b387bd1f573 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e563fe174db4cc19ae6bf8162cc3d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:02:53.073767 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337a79d7-eeb9-4f73-a8f6-5b387bd1f573 service neutron] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:53.074048 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337a79d7-eeb9-4f73-a8f6-5b387bd1f573 service neutron] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:53.074322 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337a79d7-eeb9-4f73-a8f6-5b387bd1f573 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:53.086180 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337a79d7-eeb9-4f73-a8f6-5b387bd1f573 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:02:53.088974 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337a79d7-eeb9-4f73-a8f6-5b387bd1f573 service neutron] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:02:53.107710 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1054/2113] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:02:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 11 12:02:53.623366 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa93da4e-1d87-47b9-8ab5-4cde864a72be None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:02:53.626363 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-fa93da4e-1d87-47b9-8ab5-4cde864a72be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:02:53.647823 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa93da4e-1d87-47b9-8ab5-4cde864a72be None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:02:53.668223 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5c0c6d-29c4-41d5-931c-4fb4c877cb52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:02:53.669750 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5c0c6d-29c4-41d5-931c-4fb4c877cb52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:53.671241 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5c0c6d-29c4-41d5-931c-4fb4c877cb52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:53.672072 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5c0c6d-29c4-41d5-931c-4fb4c877cb52 admin admin] PATH_INFO: `/v3/projects/deb7156d7ce349799b9040cad51098e1` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:53.683488 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5c0c6d-29c4-41d5-931c-4fb4c877cb52 admin admin] RBAC: Authorizing `identity:delete_project(project_id=deb7156d7ce349799b9040cad51098e1)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:02:53.685960 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5c0c6d-29c4-41d5-931c-4fb4c877cb52 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:02:53.769814 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-1b5c0c6d-29c4-41d5-931c-4fb4c877cb52 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'deb7156d7ce349799b9040cad51098e1', 'request_id': 'req-1b5c0c6d-29c4-41d5-931c-4fb4c877cb52'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:53.778353 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-1b5c0c6d-29c4-41d5-931c-4fb4c877cb52 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'deb7156d7ce349799b9040cad51098e1', 'request_id': 'req-1b5c0c6d-29c4-41d5-931c-4fb4c877cb52'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:53.789407 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-1b5c0c6d-29c4-41d5-931c-4fb4c877cb52 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'deb7156d7ce349799b9040cad51098e1', 'request_id': 'req-1b5c0c6d-29c4-41d5-931c-4fb4c877cb52'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:53.813970 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-1b5c0c6d-29c4-41d5-931c-4fb4c877cb52 admin admin] The token cache is being invalidate because project deb7156d7ce349799b9040cad51098e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81432) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 12:02:53.814528 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-1b5c0c6d-29c4-41d5-931c-4fb4c877cb52 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:53.815655 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-1b5c0c6d-29c4-41d5-931c-4fb4c877cb52 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:53.822631 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1060/2114] 10.222.0.59 () {58 vars in 1233 bytes} [Tue Oct 11 12:02:53 2022] DELETE /identity/v3/projects/deb7156d7ce349799b9040cad51098e1 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:02:55.292864 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337a79d7-eeb9-4f73-a8f6-5b387bd1f573 service neutron] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:02:55.294493 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-337a79d7-eeb9-4f73-a8f6-5b387bd1f573 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:02:55.308284 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337a79d7-eeb9-4f73-a8f6-5b387bd1f573 service neutron] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:02:55.378114 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98aff41-fce6-47c1-a15e-37eef4db3270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:02:55.378833 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98aff41-fce6-47c1-a15e-37eef4db3270 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:55.379115 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98aff41-fce6-47c1-a15e-37eef4db3270 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:55.379389 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98aff41-fce6-47c1-a15e-37eef4db3270 admin admin] PATH_INFO: `/v3/users/5e0688bd9f884907b7eefe56c8da8b16` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:55.380609 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98aff41-fce6-47c1-a15e-37eef4db3270 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e0688bd9f884907b7eefe56c8da8b16)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:02:55.382590 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98aff41-fce6-47c1-a15e-37eef4db3270 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:02:55.385422 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:02:55.439349 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-d98aff41-fce6-47c1-a15e-37eef4db3270 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e0688bd9f884907b7eefe56c8da8b16'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:55.453650 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-d98aff41-fce6-47c1-a15e-37eef4db3270 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e0688bd9f884907b7eefe56c8da8b16'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:55.463118 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-d98aff41-fce6-47c1-a15e-37eef4db3270 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e0688bd9f884907b7eefe56c8da8b16'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:55.463789 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-d98aff41-fce6-47c1-a15e-37eef4db3270 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e0688bd9f884907b7eefe56c8da8b16'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:55.471519 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-d98aff41-fce6-47c1-a15e-37eef4db3270 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e0688bd9f884907b7eefe56c8da8b16'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:55.477912 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1055/2115] 10.222.0.59 () {58 vars in 1224 bytes} [Tue Oct 11 12:02:55 2022] DELETE /identity/v3/users/5e0688bd9f884907b7eefe56c8da8b16 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:02:55.482619 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306ad504-0bbb-4ce4-8565-a8ef25cb6c84 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:55.483155 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306ad504-0bbb-4ce4-8565-a8ef25cb6c84 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:55.483155 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306ad504-0bbb-4ce4-8565-a8ef25cb6c84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:55.505005 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-306ad504-0bbb-4ce4-8565-a8ef25cb6c84 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:02:55.506277 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-306ad504-0bbb-4ce4-8565-a8ef25cb6c84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:02:55.572782 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1061/2116] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:02:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:02:55.584707 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98aff41-fce6-47c1-a15e-37eef4db3270 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:02:55.586241 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-d98aff41-fce6-47c1-a15e-37eef4db3270 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:02:55.666806 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98aff41-fce6-47c1-a15e-37eef4db3270 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:02:55.671752 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5995a9-03d6-46e8-ab98-1f444db71566 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e563fe174db4cc19ae6bf8162cc3d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:02:55.672385 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5995a9-03d6-46e8-ab98-1f444db71566 service neutron] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:55.672667 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5995a9-03d6-46e8-ab98-1f444db71566 service neutron] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:55.672942 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5995a9-03d6-46e8-ab98-1f444db71566 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:55.682676 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5995a9-03d6-46e8-ab98-1f444db71566 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:02:55.684106 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5995a9-03d6-46e8-ab98-1f444db71566 service neutron] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:02:55.696129 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1056/2117] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:02:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:02:56.198909 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306ad504-0bbb-4ce4-8565-a8ef25cb6c84 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:02:56.200690 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-306ad504-0bbb-4ce4-8565-a8ef25cb6c84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:02:56.223915 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306ad504-0bbb-4ce4-8565-a8ef25cb6c84 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:02:56.233179 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c778b17-097a-4ddf-a170-0eba2793b08d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:02:56.233992 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c778b17-097a-4ddf-a170-0eba2793b08d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:56.234481 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c778b17-097a-4ddf-a170-0eba2793b08d admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:56.234986 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c778b17-097a-4ddf-a170-0eba2793b08d admin admin] PATH_INFO: `/v3/projects/d6841a68b8274b5da6c8ac64035ac7b4` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:56.236311 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c778b17-097a-4ddf-a170-0eba2793b08d admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6841a68b8274b5da6c8ac64035ac7b4)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:02:56.238867 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c778b17-097a-4ddf-a170-0eba2793b08d admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:02:56.267048 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:02:56.299312 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-1c778b17-097a-4ddf-a170-0eba2793b08d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6841a68b8274b5da6c8ac64035ac7b4', 'request_id': 'req-1c778b17-097a-4ddf-a170-0eba2793b08d'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:56.306261 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-1c778b17-097a-4ddf-a170-0eba2793b08d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6841a68b8274b5da6c8ac64035ac7b4', 'request_id': 'req-1c778b17-097a-4ddf-a170-0eba2793b08d'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:56.314293 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-1c778b17-097a-4ddf-a170-0eba2793b08d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6841a68b8274b5da6c8ac64035ac7b4', 'request_id': 'req-1c778b17-097a-4ddf-a170-0eba2793b08d'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:56.329976 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-1c778b17-097a-4ddf-a170-0eba2793b08d admin admin] The token cache is being invalidate because project d6841a68b8274b5da6c8ac64035ac7b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81432) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 12:02:56.330393 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-1c778b17-097a-4ddf-a170-0eba2793b08d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:56.331099 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-1c778b17-097a-4ddf-a170-0eba2793b08d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:56.332748 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1062/2118] 10.222.0.59 () {58 vars in 1233 bytes} [Tue Oct 11 12:02:56 2022] DELETE /identity/v3/projects/d6841a68b8274b5da6c8ac64035ac7b4 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 12:02:56.337622 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5995a9-03d6-46e8-ab98-1f444db71566 service neutron] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:02:56.350377 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5995a9-03d6-46e8-ab98-1f444db71566 service neutron] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:02:56.422297 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861d6658-32a1-4b72-9db9-007877f87fc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:02:56.423153 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861d6658-32a1-4b72-9db9-007877f87fc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:56.423535 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861d6658-32a1-4b72-9db9-007877f87fc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:56.423998 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861d6658-32a1-4b72-9db9-007877f87fc6 admin admin] PATH_INFO: `/v3/users/a35edc63c4d34a7ba5f0c160eb5e6516` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:56.425618 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861d6658-32a1-4b72-9db9-007877f87fc6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a35edc63c4d34a7ba5f0c160eb5e6516)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:02:56.428822 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861d6658-32a1-4b72-9db9-007877f87fc6 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:02:56.463439 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:02:56.488453 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-861d6658-32a1-4b72-9db9-007877f87fc6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a35edc63c4d34a7ba5f0c160eb5e6516'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:56.501264 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-861d6658-32a1-4b72-9db9-007877f87fc6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a35edc63c4d34a7ba5f0c160eb5e6516'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:56.510114 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-861d6658-32a1-4b72-9db9-007877f87fc6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a35edc63c4d34a7ba5f0c160eb5e6516'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:56.510851 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-861d6658-32a1-4b72-9db9-007877f87fc6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a35edc63c4d34a7ba5f0c160eb5e6516'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:56.518883 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-861d6658-32a1-4b72-9db9-007877f87fc6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a35edc63c4d34a7ba5f0c160eb5e6516'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:56.525628 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1057/2119] 10.222.0.59 () {58 vars in 1224 bytes} [Tue Oct 11 12:02:56 2022] DELETE /identity/v3/users/a35edc63c4d34a7ba5f0c160eb5e6516 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:02:56.842233 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c778b17-097a-4ddf-a170-0eba2793b08d admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:02:56.844613 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-1c778b17-097a-4ddf-a170-0eba2793b08d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:02:56.868209 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c778b17-097a-4ddf-a170-0eba2793b08d admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:02:56.929168 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e356611-0ddf-46fc-925a-ad9fb4fa89cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:02:56.930222 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e356611-0ddf-46fc-925a-ad9fb4fa89cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:56.930609 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e356611-0ddf-46fc-925a-ad9fb4fa89cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:56.930999 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e356611-0ddf-46fc-925a-ad9fb4fa89cd admin admin] PATH_INFO: `/v3/projects/86aad02683e44551a8d83923121b56ca` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:56.932672 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e356611-0ddf-46fc-925a-ad9fb4fa89cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=86aad02683e44551a8d83923121b56ca)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:02:56.935424 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e356611-0ddf-46fc-925a-ad9fb4fa89cd admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:02:56.988408 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-8e356611-0ddf-46fc-925a-ad9fb4fa89cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86aad02683e44551a8d83923121b56ca', 'request_id': 'req-8e356611-0ddf-46fc-925a-ad9fb4fa89cd'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:56.995557 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-8e356611-0ddf-46fc-925a-ad9fb4fa89cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86aad02683e44551a8d83923121b56ca', 'request_id': 'req-8e356611-0ddf-46fc-925a-ad9fb4fa89cd'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:57.005038 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-8e356611-0ddf-46fc-925a-ad9fb4fa89cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86aad02683e44551a8d83923121b56ca', 'request_id': 'req-8e356611-0ddf-46fc-925a-ad9fb4fa89cd'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:57.019572 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-8e356611-0ddf-46fc-925a-ad9fb4fa89cd admin admin] The token cache is being invalidate because project 86aad02683e44551a8d83923121b56ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81432) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 12:02:57.020140 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-8e356611-0ddf-46fc-925a-ad9fb4fa89cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:57.021082 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-8e356611-0ddf-46fc-925a-ad9fb4fa89cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:57.023407 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1063/2120] 10.222.0.59 () {58 vars in 1233 bytes} [Tue Oct 11 12:02:56 2022] DELETE /identity/v3/projects/86aad02683e44551a8d83923121b56ca => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:02:57.028129 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861d6658-32a1-4b72-9db9-007877f87fc6 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:02:57.030584 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-861d6658-32a1-4b72-9db9-007877f87fc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:02:57.045877 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861d6658-32a1-4b72-9db9-007877f87fc6 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:02:57.115321 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e32ff6a-c672-4ae4-aa7c-5d5f9228172b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:02:57.116119 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e32ff6a-c672-4ae4-aa7c-5d5f9228172b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:57.116472 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e32ff6a-c672-4ae4-aa7c-5d5f9228172b admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:57.116815 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e32ff6a-c672-4ae4-aa7c-5d5f9228172b admin admin] PATH_INFO: `/v3/users/752435fcc1474c8f80d5cc9d825f9df3` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:57.118098 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e32ff6a-c672-4ae4-aa7c-5d5f9228172b admin admin] RBAC: Authorizing `identity:delete_user(user_id=752435fcc1474c8f80d5cc9d825f9df3)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:02:57.120256 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e32ff6a-c672-4ae4-aa7c-5d5f9228172b admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:02:57.214863 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-4e32ff6a-c672-4ae4-aa7c-5d5f9228172b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '752435fcc1474c8f80d5cc9d825f9df3'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:57.228397 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-4e32ff6a-c672-4ae4-aa7c-5d5f9228172b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '752435fcc1474c8f80d5cc9d825f9df3'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:57.242012 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-4e32ff6a-c672-4ae4-aa7c-5d5f9228172b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '752435fcc1474c8f80d5cc9d825f9df3'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:57.243202 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-4e32ff6a-c672-4ae4-aa7c-5d5f9228172b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '752435fcc1474c8f80d5cc9d825f9df3'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:57.256348 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-4e32ff6a-c672-4ae4-aa7c-5d5f9228172b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '752435fcc1474c8f80d5cc9d825f9df3'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:57.264394 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1058/2121] 10.222.0.59 () {58 vars in 1224 bytes} [Tue Oct 11 12:02:57 2022] DELETE /identity/v3/users/752435fcc1474c8f80d5cc9d825f9df3 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:02:57.948241 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e356611-0ddf-46fc-925a-ad9fb4fa89cd admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:02:57.955186 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-8e356611-0ddf-46fc-925a-ad9fb4fa89cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:02:57.985330 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e356611-0ddf-46fc-925a-ad9fb4fa89cd admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:02:58.030592 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:02:58.060196 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f71374-86e2-4d65-8e73-82e8e143ddf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:02:58.060974 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f71374-86e2-4d65-8e73-82e8e143ddf1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:58.061351 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f71374-86e2-4d65-8e73-82e8e143ddf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:58.061844 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f71374-86e2-4d65-8e73-82e8e143ddf1 admin admin] PATH_INFO: `/v3/projects/966b16a80e804c0a9ea2501af937587a` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:58.063317 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f71374-86e2-4d65-8e73-82e8e143ddf1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=966b16a80e804c0a9ea2501af937587a)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:02:58.065336 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f71374-86e2-4d65-8e73-82e8e143ddf1 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:02:58.146835 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-36f71374-86e2-4d65-8e73-82e8e143ddf1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '966b16a80e804c0a9ea2501af937587a', 'request_id': 'req-36f71374-86e2-4d65-8e73-82e8e143ddf1'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:58.155431 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-36f71374-86e2-4d65-8e73-82e8e143ddf1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '966b16a80e804c0a9ea2501af937587a', 'request_id': 'req-36f71374-86e2-4d65-8e73-82e8e143ddf1'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:58.164093 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-36f71374-86e2-4d65-8e73-82e8e143ddf1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '966b16a80e804c0a9ea2501af937587a', 'request_id': 'req-36f71374-86e2-4d65-8e73-82e8e143ddf1'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:58.180669 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-36f71374-86e2-4d65-8e73-82e8e143ddf1 admin admin] The token cache is being invalidate because project 966b16a80e804c0a9ea2501af937587a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81432) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 12:02:58.181113 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-36f71374-86e2-4d65-8e73-82e8e143ddf1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:58.181913 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-36f71374-86e2-4d65-8e73-82e8e143ddf1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:02:58.183636 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1064/2122] 10.222.0.59 () {58 vars in 1233 bytes} [Tue Oct 11 12:02:57 2022] DELETE /identity/v3/projects/966b16a80e804c0a9ea2501af937587a => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:02:58.189294 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f027214b-e48c-4277-8803-cc3822888435 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:58.190162 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f027214b-e48c-4277-8803-cc3822888435 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:58.190489 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f027214b-e48c-4277-8803-cc3822888435 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:58.219165 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-f027214b-e48c-4277-8803-cc3822888435 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:02:58.220842 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-f027214b-e48c-4277-8803-cc3822888435 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:02:58.253875 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:02:58.330533 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1059/2123] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:02:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:02:58.340411 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57679f66-f5d0-4e73-a995-6da13424b992 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:58.340714 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57679f66-f5d0-4e73-a995-6da13424b992 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:58.341047 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57679f66-f5d0-4e73-a995-6da13424b992 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:58.366800 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-57679f66-f5d0-4e73-a995-6da13424b992 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:02:58.371075 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-57679f66-f5d0-4e73-a995-6da13424b992 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:02:58.386515 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1065/2124] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:02:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:02:58.391746 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f027214b-e48c-4277-8803-cc3822888435 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:02:58.410393 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f027214b-e48c-4277-8803-cc3822888435 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:02:58.418447 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69862f4-47ec-494b-8d15-bbe7d1d8222f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:02:58.420015 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69862f4-47ec-494b-8d15-bbe7d1d8222f admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:58.420015 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69862f4-47ec-494b-8d15-bbe7d1d8222f admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:58.420222 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69862f4-47ec-494b-8d15-bbe7d1d8222f admin admin] PATH_INFO: `/v3/domains` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:58.421078 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69862f4-47ec-494b-8d15-bbe7d1d8222f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 12:02:58.421400 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69862f4-47ec-494b-8d15-bbe7d1d8222f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:02:58.423154 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69862f4-47ec-494b-8d15-bbe7d1d8222f admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:02:58.438802 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1060/2125] 10.222.0.59 () {58 vars in 1153 bytes} [Tue Oct 11 12:02:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 12:02:58.441419 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57679f66-f5d0-4e73-a995-6da13424b992 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:02:58.454608 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57679f66-f5d0-4e73-a995-6da13424b992 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:02:58.460039 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42c525e-951b-4fbc-87c3-815f4772f5bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:02:58.460741 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42c525e-951b-4fbc-87c3-815f4772f5bb admin admin] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:58.461830 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42c525e-951b-4fbc-87c3-815f4772f5bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:58.461830 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42c525e-951b-4fbc-87c3-815f4772f5bb admin admin] PATH_INFO: `/v3/projects` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:58.462155 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42c525e-951b-4fbc-87c3-815f4772f5bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:02:58.464654 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42c525e-951b-4fbc-87c3-815f4772f5bb admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:02:58.502483 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1066/2126] 10.222.0.59 () {60 vars in 1153 bytes} [Tue Oct 11 12:02:58 2022] POST /identity/v3/projects => generated 373 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:02:58.504035 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69862f4-47ec-494b-8d15-bbe7d1d8222f admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:02:58.514847 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69862f4-47ec-494b-8d15-bbe7d1d8222f admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:02:58.589887 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df0c0ed-30c6-4a73-ab70-4600ee0040e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:02:58.590449 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df0c0ed-30c6-4a73-ab70-4600ee0040e0 admin admin] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:58.590784 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df0c0ed-30c6-4a73-ab70-4600ee0040e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:58.591040 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df0c0ed-30c6-4a73-ab70-4600ee0040e0 admin admin] PATH_INFO: `/v3/users` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:58.592090 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df0c0ed-30c6-4a73-ab70-4600ee0040e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:02:58.594330 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df0c0ed-30c6-4a73-ab70-4600ee0040e0 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:02:58.621866 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1061/2127] 10.222.0.59 () {60 vars in 1144 bytes} [Tue Oct 11 12:02:58 2022] POST /identity/v3/users => generated 287 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 12:02:58.627634 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42c525e-951b-4fbc-87c3-815f4772f5bb admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:02:58.642946 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42c525e-951b-4fbc-87c3-815f4772f5bb admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:02:58.651742 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a4f3f7-287d-451a-9bc0-a4d154cb1219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:02:58.652819 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a4f3f7-287d-451a-9bc0-a4d154cb1219 admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:58.653334 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a4f3f7-287d-451a-9bc0-a4d154cb1219 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:58.653840 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a4f3f7-287d-451a-9bc0-a4d154cb1219 admin admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:58.655170 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a4f3f7-287d-451a-9bc0-a4d154cb1219 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:02:58.658174 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a4f3f7-287d-451a-9bc0-a4d154cb1219 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:02:58.678625 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1067/2128] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:02:58 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:02:58.683783 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df0c0ed-30c6-4a73-ab70-4600ee0040e0 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:02:58.695687 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df0c0ed-30c6-4a73-ab70-4600ee0040e0 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:02:58.701990 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b169a44c-2b14-459e-9947-79f467479684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:02:58.702881 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b169a44c-2b14-459e-9947-79f467479684 admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:58.703257 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b169a44c-2b14-459e-9947-79f467479684 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:58.703705 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b169a44c-2b14-459e-9947-79f467479684 admin admin] PATH_INFO: `/v3/roles` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:58.704755 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b169a44c-2b14-459e-9947-79f467479684 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:02:58.708105 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b169a44c-2b14-459e-9947-79f467479684 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:02:58.724502 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1062/2129] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:02:58 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 11 12:02:58.729683 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a4f3f7-287d-451a-9bc0-a4d154cb1219 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:02:58.741475 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a4f3f7-287d-451a-9bc0-a4d154cb1219 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:02:58.748193 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd313261-0db6-4fec-b978-a925f3f1d44e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:02:58.748931 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd313261-0db6-4fec-b978-a925f3f1d44e admin admin] REQUEST_METHOD: `PUT` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:58.749272 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd313261-0db6-4fec-b978-a925f3f1d44e admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:58.749547 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd313261-0db6-4fec-b978-a925f3f1d44e admin admin] PATH_INFO: `/v3/projects/66d90ea7f97c48b4b1c026fedd768ae8/users/c336c1557b834864a1f2439f057d80f0/roles/de2d2bd26fff4748a79990f97a107164` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:58.770025 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd313261-0db6-4fec-b978-a925f3f1d44e admin admin] RBAC: Authorizing `identity:create_grant(project_id=66d90ea7f97c48b4b1c026fedd768ae8, user_id=c336c1557b834864a1f2439f057d80f0, role_id=de2d2bd26fff4748a79990f97a107164)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:02:58.772787 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd313261-0db6-4fec-b978-a925f3f1d44e admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:02:58.794418 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1068/2130] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 12:02:58 2022] PUT /identity/v3/projects/66d90ea7f97c48b4b1c026fedd768ae8/users/c336c1557b834864a1f2439f057d80f0/roles/de2d2bd26fff4748a79990f97a107164 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:02:58.805517 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531db173-dbba-417b-92ae-f5ec0ff06dff None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:58.805945 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531db173-dbba-417b-92ae-f5ec0ff06dff None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:58.806318 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531db173-dbba-417b-92ae-f5ec0ff06dff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:58.838622 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-531db173-dbba-417b-92ae-f5ec0ff06dff None None] MFA Rules not processed for user `c336c1557b834864a1f2439f057d80f0`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:02:58.840047 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-531db173-dbba-417b-92ae-f5ec0ff06dff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:02:58.925139 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1063/2131] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:02:58 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 12:02:58.937378 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd313261-0db6-4fec-b978-a925f3f1d44e admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:02:58.939960 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-cd313261-0db6-4fec-b978-a925f3f1d44e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:02:59.040749 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd313261-0db6-4fec-b978-a925f3f1d44e admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:02:59.046201 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ecdaf5-50d1-4410-9a17-bc263ad10778 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b1b651c73664d08accc6edf2f79bace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:02:59.046892 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ecdaf5-50d1-4410-9a17-bc263ad10778 service cinder] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:02:59.047297 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ecdaf5-50d1-4410-9a17-bc263ad10778 service cinder] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:02:59.047476 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ecdaf5-50d1-4410-9a17-bc263ad10778 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:02:59.059044 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ecdaf5-50d1-4410-9a17-bc263ad10778 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:02:59.060588 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ecdaf5-50d1-4410-9a17-bc263ad10778 service cinder] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:02:59.074513 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1069/2132] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:02:58 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:03:00.693826 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f635ea8-a160-47e0-8d39-5f3661ec9e9c None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:03:00.694699 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f635ea8-a160-47e0-8d39-5f3661ec9e9c None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:03:00.694699 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f635ea8-a160-47e0-8d39-5f3661ec9e9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:03:00.719826 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-4f635ea8-a160-47e0-8d39-5f3661ec9e9c None None] MFA Rules not processed for user `c336c1557b834864a1f2439f057d80f0`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:03:00.721110 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-4f635ea8-a160-47e0-8d39-5f3661ec9e9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:03:00.728646 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1064/2133] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:03:00 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:03:03.300124 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7862eb31-62a0-4aa4-89dc-d90bb587cb25 None None] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:03:03.301082 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7862eb31-62a0-4aa4-89dc-d90bb587cb25 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:03:03.301266 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7862eb31-62a0-4aa4-89dc-d90bb587cb25 None None] PATH_INFO: `/` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:03:03.303190 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1070/2134] 10.222.0.59 () {54 vars in 961 bytes} [Tue Oct 11 12:03:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 12:03:03.313641 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e6dc00-8115-465d-bf7d-9050885dcfbc None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:03:03.314628 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e6dc00-8115-465d-bf7d-9050885dcfbc None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:03:03.314628 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e6dc00-8115-465d-bf7d-9050885dcfbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:03:03.340202 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-25e6dc00-8115-465d-bf7d-9050885dcfbc None None] MFA Rules not processed for user `f272cdd1bcc04b4e8261ed533cfd4fd7`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:03:03.341708 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-25e6dc00-8115-465d-bf7d-9050885dcfbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:03:03.347990 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:03:03.425980 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1065/2135] 10.222.0.59 () {58 vars in 1058 bytes} [Tue Oct 11 12:03:03 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:03:03.434949 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7862eb31-62a0-4aa4-89dc-d90bb587cb25 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:03:03.475680 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7862eb31-62a0-4aa4-89dc-d90bb587cb25 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:03:03.487929 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254b6fca-7de2-4c54-86bd-fe555815839b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f272cdd1bcc04b4e8261ed533cfd4fd7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:03:03.489575 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254b6fca-7de2-4c54-86bd-fe555815839b service placement] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:03:03.489859 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254b6fca-7de2-4c54-86bd-fe555815839b service placement] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:03:03.490141 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254b6fca-7de2-4c54-86bd-fe555815839b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:03:03.492582 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-254b6fca-7de2-4c54-86bd-fe555815839b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:03:03.503711 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:03:03.510352 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254b6fca-7de2-4c54-86bd-fe555815839b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:03:03.513165 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254b6fca-7de2-4c54-86bd-fe555815839b service placement] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:03:03.527701 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1071/2136] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:03:03 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 11 12:03:13.414657 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=24 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:03:13.529536 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=3 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:03:15.124803 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5e7868-10d5-44ef-a5e0-30a83dbae89b None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:03:15.125761 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5e7868-10d5-44ef-a5e0-30a83dbae89b None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:03:15.125761 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5e7868-10d5-44ef-a5e0-30a83dbae89b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:03:15.155696 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-cd5e7868-10d5-44ef-a5e0-30a83dbae89b None None] MFA Rules not processed for user `8a19eceb62ce4157a4c5fe1866a89df9`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:03:15.157248 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-cd5e7868-10d5-44ef-a5e0-30a83dbae89b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:03:15.239711 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1066/2137] 10.222.0.59 () {58 vars in 1009 bytes} [Tue Oct 11 12:03:15 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 12:03:15.290861 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254b6fca-7de2-4c54-86bd-fe555815839b service placement] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:03:15.292838 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-254b6fca-7de2-4c54-86bd-fe555815839b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:03:15.379511 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254b6fca-7de2-4c54-86bd-fe555815839b service placement] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:03:15.385942 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c30c466-68ee-4c96-a65b-260ff445786e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76e7fee1993a40d5ba4295e7daeb3e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:03:15.386739 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c30c466-68ee-4c96-a65b-260ff445786e service swift] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:03:15.387080 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c30c466-68ee-4c96-a65b-260ff445786e service swift] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:03:15.387340 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c30c466-68ee-4c96-a65b-260ff445786e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:03:15.398602 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c30c466-68ee-4c96-a65b-260ff445786e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:03:15.400617 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c30c466-68ee-4c96-a65b-260ff445786e service swift] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:03:15.411768 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1072/2138] 10.222.0.59 () {60 vars in 1384 bytes} [Tue Oct 11 12:03:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 11 12:03:25.231766 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=25 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:03:25.406588 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=32 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:03:28.283551 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a86d995-824c-448e-b047-4c7eaf0acb73 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:03:28.284197 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a86d995-824c-448e-b047-4c7eaf0acb73 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:03:28.284296 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a86d995-824c-448e-b047-4c7eaf0acb73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:03:28.307364 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-1a86d995-824c-448e-b047-4c7eaf0acb73 None None] MFA Rules not processed for user `8a19eceb62ce4157a4c5fe1866a89df9`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:03:28.308647 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-1a86d995-824c-448e-b047-4c7eaf0acb73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:03:28.317572 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1067/2139] 10.222.0.59 () {58 vars in 1009 bytes} [Tue Oct 11 12:03:28 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:03:28.340089 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c30c466-68ee-4c96-a65b-260ff445786e service swift] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:03:28.357506 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c30c466-68ee-4c96-a65b-260ff445786e service swift] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:03:28.367410 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b76ef0a-c517-4163-8965-46f1686ca872 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76e7fee1993a40d5ba4295e7daeb3e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:03:28.368616 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b76ef0a-c517-4163-8965-46f1686ca872 service swift] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:03:28.368904 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b76ef0a-c517-4163-8965-46f1686ca872 service swift] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:03:28.369207 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b76ef0a-c517-4163-8965-46f1686ca872 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:03:28.379242 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b76ef0a-c517-4163-8965-46f1686ca872 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:03:28.380890 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b76ef0a-c517-4163-8965-46f1686ca872 service swift] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:03:28.390796 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1073/2140] 10.222.0.59 () {60 vars in 1384 bytes} [Tue Oct 11 12:03:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 11 12:03:29.293634 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e6df9f-a9ec-4166-87f7-3f03743ee6e4 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:03:29.294842 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e6df9f-a9ec-4166-87f7-3f03743ee6e4 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:03:29.294842 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e6df9f-a9ec-4166-87f7-3f03743ee6e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:03:29.325550 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-63e6df9f-a9ec-4166-87f7-3f03743ee6e4 None None] MFA Rules not processed for user `8a19eceb62ce4157a4c5fe1866a89df9`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:03:29.327728 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-63e6df9f-a9ec-4166-87f7-3f03743ee6e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:03:29.335717 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1068/2141] 10.222.0.59 () {58 vars in 967 bytes} [Tue Oct 11 12:03:29 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:03:29.362115 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b76ef0a-c517-4163-8965-46f1686ca872 service swift] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:03:29.377026 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b76ef0a-c517-4163-8965-46f1686ca872 service swift] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:03:29.382924 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906f4264-8a75-4264-a8e1-bf7499d3761c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76e7fee1993a40d5ba4295e7daeb3e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:03:29.383544 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906f4264-8a75-4264-a8e1-bf7499d3761c service swift] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:03:29.383818 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906f4264-8a75-4264-a8e1-bf7499d3761c service swift] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:03:29.384268 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906f4264-8a75-4264-a8e1-bf7499d3761c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:03:29.394194 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906f4264-8a75-4264-a8e1-bf7499d3761c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:03:29.396378 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906f4264-8a75-4264-a8e1-bf7499d3761c service swift] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:03:29.408277 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1074/2142] 10.222.0.59 () {60 vars in 1384 bytes} [Tue Oct 11 12:03:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 11 12:03:29.695853 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e80e48-17e3-46ef-9097-9a5315f3f19c None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:03:29.696360 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e80e48-17e3-46ef-9097-9a5315f3f19c None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:03:29.696811 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e80e48-17e3-46ef-9097-9a5315f3f19c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:03:29.719024 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-d2e80e48-17e3-46ef-9097-9a5315f3f19c None None] MFA Rules not processed for user `8a19eceb62ce4157a4c5fe1866a89df9`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:03:29.720789 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-d2e80e48-17e3-46ef-9097-9a5315f3f19c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:03:29.728563 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1069/2143] 10.222.0.59 () {58 vars in 1009 bytes} [Tue Oct 11 12:03:29 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:03:29.745607 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906f4264-8a75-4264-a8e1-bf7499d3761c service swift] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:03:29.758645 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906f4264-8a75-4264-a8e1-bf7499d3761c service swift] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:03:29.765459 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6ce118-7b92-4fb3-b618-f12652a248ce service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76e7fee1993a40d5ba4295e7daeb3e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:03:29.766357 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6ce118-7b92-4fb3-b618-f12652a248ce service swift] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:03:29.766767 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6ce118-7b92-4fb3-b618-f12652a248ce service swift] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:03:29.767135 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6ce118-7b92-4fb3-b618-f12652a248ce service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:03:29.777631 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6ce118-7b92-4fb3-b618-f12652a248ce service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:03:29.779525 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6ce118-7b92-4fb3-b618-f12652a248ce service swift] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:03:29.789922 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1075/2144] 10.222.0.59 () {60 vars in 1384 bytes} [Tue Oct 11 12:03:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 11 12:03:30.289660 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec2bed0-3173-4d76-8edc-9480ef058ea8 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:03:30.289942 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec2bed0-3173-4d76-8edc-9480ef058ea8 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:03:30.301873 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec2bed0-3173-4d76-8edc-9480ef058ea8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:03:30.318816 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-3ec2bed0-3173-4d76-8edc-9480ef058ea8 None None] MFA Rules not processed for user `8a19eceb62ce4157a4c5fe1866a89df9`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:03:30.320234 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-3ec2bed0-3173-4d76-8edc-9480ef058ea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:03:30.351340 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1070/2145] 10.222.0.59 () {58 vars in 967 bytes} [Tue Oct 11 12:03:30 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:03:30.371531 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6ce118-7b92-4fb3-b618-f12652a248ce service swift] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:03:30.387089 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6ce118-7b92-4fb3-b618-f12652a248ce service swift] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:03:30.397315 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7639416f-b921-4db3-aaaa-a5022b50e3e2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76e7fee1993a40d5ba4295e7daeb3e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:03:30.397959 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7639416f-b921-4db3-aaaa-a5022b50e3e2 service swift] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:03:30.398441 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7639416f-b921-4db3-aaaa-a5022b50e3e2 service swift] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:03:30.398759 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7639416f-b921-4db3-aaaa-a5022b50e3e2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:03:30.411107 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7639416f-b921-4db3-aaaa-a5022b50e3e2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:03:30.413325 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7639416f-b921-4db3-aaaa-a5022b50e3e2 service swift] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:03:30.427152 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1076/2146] 10.222.0.59 () {60 vars in 1384 bytes} [Tue Oct 11 12:03:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 11 12:03:32.873701 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c559e8e0-53aa-4910-9398-0cb3f157af51 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:03:32.874203 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c559e8e0-53aa-4910-9398-0cb3f157af51 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:03:32.874203 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c559e8e0-53aa-4910-9398-0cb3f157af51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:03:32.897917 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-c559e8e0-53aa-4910-9398-0cb3f157af51 None None] MFA Rules not processed for user `8a19eceb62ce4157a4c5fe1866a89df9`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:03:32.899638 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-c559e8e0-53aa-4910-9398-0cb3f157af51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:03:32.914139 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1071/2147] 10.222.0.59 () {58 vars in 967 bytes} [Tue Oct 11 12:03:32 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:03:32.928094 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7639416f-b921-4db3-aaaa-a5022b50e3e2 service swift] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:03:32.939551 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7639416f-b921-4db3-aaaa-a5022b50e3e2 service swift] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:03:32.947360 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77fb708-13c1-4a19-a9c9-4e503bc0c87b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76e7fee1993a40d5ba4295e7daeb3e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:03:32.948361 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77fb708-13c1-4a19-a9c9-4e503bc0c87b service swift] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:03:32.948745 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77fb708-13c1-4a19-a9c9-4e503bc0c87b service swift] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:03:32.949100 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77fb708-13c1-4a19-a9c9-4e503bc0c87b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:03:32.959383 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77fb708-13c1-4a19-a9c9-4e503bc0c87b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:03:32.961032 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77fb708-13c1-4a19-a9c9-4e503bc0c87b service swift] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:03:32.972054 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1077/2148] 10.222.0.59 () {60 vars in 1384 bytes} [Tue Oct 11 12:03:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 11 12:03:33.934938 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c559e8e0-53aa-4910-9398-0cb3f157af51 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:03:34.008131 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c559e8e0-53aa-4910-9398-0cb3f157af51 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:03:34.012875 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c647915-a891-45b8-b12f-dfe38b12107e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:03:34.013511 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c647915-a891-45b8-b12f-dfe38b12107e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:03:34.013793 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c647915-a891-45b8-b12f-dfe38b12107e admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:03:34.014067 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c647915-a891-45b8-b12f-dfe38b12107e admin admin] PATH_INFO: `/v3/users/c336c1557b834864a1f2439f057d80f0` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:03:34.015255 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c647915-a891-45b8-b12f-dfe38b12107e admin admin] RBAC: Authorizing `identity:delete_user(user_id=c336c1557b834864a1f2439f057d80f0)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:03:34.017213 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c647915-a891-45b8-b12f-dfe38b12107e admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:03:34.070759 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-4c647915-a891-45b8-b12f-dfe38b12107e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c336c1557b834864a1f2439f057d80f0'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:03:34.088054 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-4c647915-a891-45b8-b12f-dfe38b12107e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c336c1557b834864a1f2439f057d80f0'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:03:34.097419 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-4c647915-a891-45b8-b12f-dfe38b12107e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c336c1557b834864a1f2439f057d80f0'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:03:34.098522 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-4c647915-a891-45b8-b12f-dfe38b12107e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c336c1557b834864a1f2439f057d80f0'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:03:34.106180 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-4c647915-a891-45b8-b12f-dfe38b12107e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c336c1557b834864a1f2439f057d80f0'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:03:34.117232 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1072/2149] 10.222.0.59 () {58 vars in 1224 bytes} [Tue Oct 11 12:03:33 2022] DELETE /identity/v3/users/c336c1557b834864a1f2439f057d80f0 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:03:34.124420 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8379f044-9867-43a8-adcb-a9ab2fcfdb6e None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:03:34.124966 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8379f044-9867-43a8-adcb-a9ab2fcfdb6e None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:03:34.125495 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8379f044-9867-43a8-adcb-a9ab2fcfdb6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:03:34.155377 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-8379f044-9867-43a8-adcb-a9ab2fcfdb6e None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:03:34.156612 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-8379f044-9867-43a8-adcb-a9ab2fcfdb6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:03:34.229113 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1078/2150] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:03:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:03:34.239589 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c647915-a891-45b8-b12f-dfe38b12107e admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:03:34.241179 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-4c647915-a891-45b8-b12f-dfe38b12107e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:03:34.299848 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:03:34.306581 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c647915-a891-45b8-b12f-dfe38b12107e admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:03:34.311815 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2f0ee3-60e4-462a-b4f3-208058eb74ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e563fe174db4cc19ae6bf8162cc3d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:03:34.312452 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2f0ee3-60e4-462a-b4f3-208058eb74ba service neutron] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:03:34.312749 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2f0ee3-60e4-462a-b4f3-208058eb74ba service neutron] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:03:34.313028 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2f0ee3-60e4-462a-b4f3-208058eb74ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:03:34.323725 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2f0ee3-60e4-462a-b4f3-208058eb74ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:03:34.325262 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2f0ee3-60e4-462a-b4f3-208058eb74ba service neutron] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:03:34.335971 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1073/2151] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:03:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:03:34.533593 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8379f044-9867-43a8-adcb-a9ab2fcfdb6e None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:03:34.535563 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-8379f044-9867-43a8-adcb-a9ab2fcfdb6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:03:34.556093 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8379f044-9867-43a8-adcb-a9ab2fcfdb6e None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:03:34.569826 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc82b58-2ddf-4c5d-b257-866b2b7e3c35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:03:34.570503 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc82b58-2ddf-4c5d-b257-866b2b7e3c35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:03:34.570859 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc82b58-2ddf-4c5d-b257-866b2b7e3c35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:03:34.571093 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc82b58-2ddf-4c5d-b257-866b2b7e3c35 admin admin] PATH_INFO: `/v3/projects/66d90ea7f97c48b4b1c026fedd768ae8` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:03:34.572253 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc82b58-2ddf-4c5d-b257-866b2b7e3c35 admin admin] RBAC: Authorizing `identity:delete_project(project_id=66d90ea7f97c48b4b1c026fedd768ae8)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:03:34.574212 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc82b58-2ddf-4c5d-b257-866b2b7e3c35 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:03:34.625830 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-fcc82b58-2ddf-4c5d-b257-866b2b7e3c35 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66d90ea7f97c48b4b1c026fedd768ae8', 'request_id': 'req-fcc82b58-2ddf-4c5d-b257-866b2b7e3c35'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:03:34.633058 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-fcc82b58-2ddf-4c5d-b257-866b2b7e3c35 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66d90ea7f97c48b4b1c026fedd768ae8', 'request_id': 'req-fcc82b58-2ddf-4c5d-b257-866b2b7e3c35'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:03:34.639660 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-fcc82b58-2ddf-4c5d-b257-866b2b7e3c35 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66d90ea7f97c48b4b1c026fedd768ae8', 'request_id': 'req-fcc82b58-2ddf-4c5d-b257-866b2b7e3c35'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:03:34.653705 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-fcc82b58-2ddf-4c5d-b257-866b2b7e3c35 admin admin] The token cache is being invalidate because project 66d90ea7f97c48b4b1c026fedd768ae8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81432) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 12:03:34.653997 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-fcc82b58-2ddf-4c5d-b257-866b2b7e3c35 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:03:34.654666 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-fcc82b58-2ddf-4c5d-b257-866b2b7e3c35 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:03:34.656339 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1079/2152] 10.222.0.59 () {58 vars in 1233 bytes} [Tue Oct 11 12:03:34 2022] DELETE /identity/v3/projects/66d90ea7f97c48b4b1c026fedd768ae8 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:03:35.074314 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2f0ee3-60e4-462a-b4f3-208058eb74ba service neutron] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:03:35.075904 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-2a2f0ee3-60e4-462a-b4f3-208058eb74ba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:03:35.090178 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2f0ee3-60e4-462a-b4f3-208058eb74ba service neutron] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:03:35.150594 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb54102-7bf8-4e6c-b14c-0542b857814f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:03:35.151267 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb54102-7bf8-4e6c-b14c-0542b857814f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:03:35.151536 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb54102-7bf8-4e6c-b14c-0542b857814f admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:03:35.151826 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb54102-7bf8-4e6c-b14c-0542b857814f admin admin] PATH_INFO: `/v3/users/ae372b81003e41aeb654da0bcca928d5` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:03:35.153004 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb54102-7bf8-4e6c-b14c-0542b857814f admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae372b81003e41aeb654da0bcca928d5)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:03:35.155118 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb54102-7bf8-4e6c-b14c-0542b857814f admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:03:35.205078 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-bbb54102-7bf8-4e6c-b14c-0542b857814f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae372b81003e41aeb654da0bcca928d5'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:03:35.218122 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-bbb54102-7bf8-4e6c-b14c-0542b857814f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae372b81003e41aeb654da0bcca928d5'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:03:35.226701 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-bbb54102-7bf8-4e6c-b14c-0542b857814f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae372b81003e41aeb654da0bcca928d5'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:03:35.227416 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-bbb54102-7bf8-4e6c-b14c-0542b857814f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae372b81003e41aeb654da0bcca928d5'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:03:35.235702 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-bbb54102-7bf8-4e6c-b14c-0542b857814f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae372b81003e41aeb654da0bcca928d5'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:03:35.242079 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1074/2153] 10.222.0.59 () {58 vars in 1224 bytes} [Tue Oct 11 12:03:35 2022] DELETE /identity/v3/users/ae372b81003e41aeb654da0bcca928d5 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:03:35.425495 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc82b58-2ddf-4c5d-b257-866b2b7e3c35 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:03:35.427433 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-fcc82b58-2ddf-4c5d-b257-866b2b7e3c35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:03:35.443869 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc82b58-2ddf-4c5d-b257-866b2b7e3c35 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:03:35.457715 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:03:35.505675 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a04e93-bd93-4724-87fb-620cb6488093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:03:35.506251 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a04e93-bd93-4724-87fb-620cb6488093 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:03:35.506535 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a04e93-bd93-4724-87fb-620cb6488093 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:03:35.506883 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a04e93-bd93-4724-87fb-620cb6488093 admin admin] PATH_INFO: `/v3/projects/f943e2a175344e40b29c72019ba95b03` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:03:35.508178 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a04e93-bd93-4724-87fb-620cb6488093 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f943e2a175344e40b29c72019ba95b03)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:03:35.510192 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a04e93-bd93-4724-87fb-620cb6488093 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:03:35.571128 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-65a04e93-bd93-4724-87fb-620cb6488093 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f943e2a175344e40b29c72019ba95b03', 'request_id': 'req-65a04e93-bd93-4724-87fb-620cb6488093'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:03:35.577169 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-65a04e93-bd93-4724-87fb-620cb6488093 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f943e2a175344e40b29c72019ba95b03', 'request_id': 'req-65a04e93-bd93-4724-87fb-620cb6488093'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:03:35.583760 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-65a04e93-bd93-4724-87fb-620cb6488093 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f943e2a175344e40b29c72019ba95b03', 'request_id': 'req-65a04e93-bd93-4724-87fb-620cb6488093'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:03:35.600529 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-65a04e93-bd93-4724-87fb-620cb6488093 admin admin] The token cache is being invalidate because project f943e2a175344e40b29c72019ba95b03 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81432) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 12:03:35.600913 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-65a04e93-bd93-4724-87fb-620cb6488093 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:03:35.601603 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-65a04e93-bd93-4724-87fb-620cb6488093 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:03:35.603851 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1080/2154] 10.222.0.59 () {58 vars in 1233 bytes} [Tue Oct 11 12:03:35 2022] DELETE /identity/v3/projects/f943e2a175344e40b29c72019ba95b03 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:03:35.610611 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0a5bfb-41b8-41aa-8caa-8a213b8eabb9 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:03:35.611147 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0a5bfb-41b8-41aa-8caa-8a213b8eabb9 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:03:35.611447 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0a5bfb-41b8-41aa-8caa-8a213b8eabb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:03:35.636682 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-ad0a5bfb-41b8-41aa-8caa-8a213b8eabb9 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:03:35.637938 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-ad0a5bfb-41b8-41aa-8caa-8a213b8eabb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:03:35.704272 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1075/2155] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:03:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:03:35.713061 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1c7ba5-24b1-4729-be9e-26edb7565ba6 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:03:35.713430 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1c7ba5-24b1-4729-be9e-26edb7565ba6 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:03:35.713737 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1c7ba5-24b1-4729-be9e-26edb7565ba6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:03:35.741628 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-cd1c7ba5-24b1-4729-be9e-26edb7565ba6 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:03:35.742894 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-cd1c7ba5-24b1-4729-be9e-26edb7565ba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:03:35.751576 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1081/2156] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:03:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 12:03:35.756321 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0a5bfb-41b8-41aa-8caa-8a213b8eabb9 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:03:35.766714 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0a5bfb-41b8-41aa-8caa-8a213b8eabb9 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:03:35.772052 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bf5dc0-0718-462a-b54c-37279b9c88d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:03:35.772694 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bf5dc0-0718-462a-b54c-37279b9c88d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:03:35.772970 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bf5dc0-0718-462a-b54c-37279b9c88d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:03:35.773210 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bf5dc0-0718-462a-b54c-37279b9c88d2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:03:35.773937 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bf5dc0-0718-462a-b54c-37279b9c88d2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 12:03:35.774602 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bf5dc0-0718-462a-b54c-37279b9c88d2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:03:35.776284 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bf5dc0-0718-462a-b54c-37279b9c88d2 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:03:35.779358 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:03:35.792777 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1076/2157] 10.222.0.59 () {58 vars in 1153 bytes} [Tue Oct 11 12:03:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 12:03:35.797980 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1c7ba5-24b1-4729-be9e-26edb7565ba6 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:03:35.808247 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1c7ba5-24b1-4729-be9e-26edb7565ba6 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:03:35.814280 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c61285d-c648-4ed9-a275-7e5cd55c7c8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:03:35.815306 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c61285d-c648-4ed9-a275-7e5cd55c7c8a admin admin] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:03:35.815589 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c61285d-c648-4ed9-a275-7e5cd55c7c8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:03:35.815863 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c61285d-c648-4ed9-a275-7e5cd55c7c8a admin admin] PATH_INFO: `/v3/projects` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:03:35.816584 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c61285d-c648-4ed9-a275-7e5cd55c7c8a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:03:35.818611 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c61285d-c648-4ed9-a275-7e5cd55c7c8a admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:03:35.857069 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1082/2158] 10.222.0.59 () {60 vars in 1153 bytes} [Tue Oct 11 12:03:35 2022] POST /identity/v3/projects => generated 381 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:03:35.858674 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bf5dc0-0718-462a-b54c-37279b9c88d2 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:03:35.869567 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bf5dc0-0718-462a-b54c-37279b9c88d2 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:03:35.937034 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7404a6b0-17eb-4ba2-b43a-24aeb3e5c18d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:03:35.937655 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7404a6b0-17eb-4ba2-b43a-24aeb3e5c18d admin admin] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:03:35.937961 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7404a6b0-17eb-4ba2-b43a-24aeb3e5c18d admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:03:35.938235 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7404a6b0-17eb-4ba2-b43a-24aeb3e5c18d admin admin] PATH_INFO: `/v3/users` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:03:35.939196 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7404a6b0-17eb-4ba2-b43a-24aeb3e5c18d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:03:35.941150 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7404a6b0-17eb-4ba2-b43a-24aeb3e5c18d admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:03:35.966352 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1077/2159] 10.222.0.59 () {60 vars in 1144 bytes} [Tue Oct 11 12:03:35 2022] POST /identity/v3/users => generated 291 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:03:35.971673 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c61285d-c648-4ed9-a275-7e5cd55c7c8a admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:03:35.982971 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c61285d-c648-4ed9-a275-7e5cd55c7c8a admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:03:35.988405 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf06d2a-5f16-40c8-bda9-0804c5e78b01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:03:35.989017 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf06d2a-5f16-40c8-bda9-0804c5e78b01 admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:03:35.989365 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf06d2a-5f16-40c8-bda9-0804c5e78b01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:03:35.989667 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf06d2a-5f16-40c8-bda9-0804c5e78b01 admin admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:03:35.990317 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf06d2a-5f16-40c8-bda9-0804c5e78b01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:03:35.992114 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf06d2a-5f16-40c8-bda9-0804c5e78b01 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:03:36.004843 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1083/2160] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:03:35 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:03:36.009324 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7404a6b0-17eb-4ba2-b43a-24aeb3e5c18d admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:03:36.020523 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7404a6b0-17eb-4ba2-b43a-24aeb3e5c18d admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:03:36.029860 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01041dc-3324-47af-b778-a2a9f2f7c652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:03:36.030346 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01041dc-3324-47af-b778-a2a9f2f7c652 admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:03:36.030651 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01041dc-3324-47af-b778-a2a9f2f7c652 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:03:36.030923 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01041dc-3324-47af-b778-a2a9f2f7c652 admin admin] PATH_INFO: `/v3/roles` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:03:36.031579 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01041dc-3324-47af-b778-a2a9f2f7c652 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:03:36.033188 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01041dc-3324-47af-b778-a2a9f2f7c652 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:03:36.044299 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1078/2161] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:03:36 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:03:36.048725 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf06d2a-5f16-40c8-bda9-0804c5e78b01 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:03:36.059190 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf06d2a-5f16-40c8-bda9-0804c5e78b01 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:03:36.064722 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592a923d-7f0b-4854-b95d-a421936e57b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:03:36.065382 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592a923d-7f0b-4854-b95d-a421936e57b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:03:36.065656 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592a923d-7f0b-4854-b95d-a421936e57b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:03:36.065925 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592a923d-7f0b-4854-b95d-a421936e57b4 admin admin] PATH_INFO: `/v3/projects/6b5bdf17f2924e7a8a3ea07b707404aa/users/b5daeff9822142978b250c72adcdcfc6/roles/de2d2bd26fff4748a79990f97a107164` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:03:36.082641 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592a923d-7f0b-4854-b95d-a421936e57b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b5bdf17f2924e7a8a3ea07b707404aa, user_id=b5daeff9822142978b250c72adcdcfc6, role_id=de2d2bd26fff4748a79990f97a107164)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:03:36.085322 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592a923d-7f0b-4854-b95d-a421936e57b4 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:03:36.100174 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1084/2162] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 12:03:36 2022] PUT /identity/v3/projects/6b5bdf17f2924e7a8a3ea07b707404aa/users/b5daeff9822142978b250c72adcdcfc6/roles/de2d2bd26fff4748a79990f97a107164 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:03:36.111339 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f2e601-9401-4381-9a77-d1649be6452d None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:03:36.111962 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f2e601-9401-4381-9a77-d1649be6452d None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:03:36.111962 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f2e601-9401-4381-9a77-d1649be6452d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:03:36.143570 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-26f2e601-9401-4381-9a77-d1649be6452d None None] MFA Rules not processed for user `b5daeff9822142978b250c72adcdcfc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:03:36.144806 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-26f2e601-9401-4381-9a77-d1649be6452d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:03:36.217805 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1079/2163] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:03:36 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:03:36.232501 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592a923d-7f0b-4854-b95d-a421936e57b4 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:03:36.235930 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-592a923d-7f0b-4854-b95d-a421936e57b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:03:36.287381 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:03:36.299686 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592a923d-7f0b-4854-b95d-a421936e57b4 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:03:36.303609 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612142b7-ad9d-4a79-beae-bc14a74f6417 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b1b651c73664d08accc6edf2f79bace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:03:36.304215 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612142b7-ad9d-4a79-beae-bc14a74f6417 service cinder] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:03:36.304490 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612142b7-ad9d-4a79-beae-bc14a74f6417 service cinder] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:03:36.304761 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612142b7-ad9d-4a79-beae-bc14a74f6417 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:03:36.313109 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612142b7-ad9d-4a79-beae-bc14a74f6417 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:03:36.314614 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612142b7-ad9d-4a79-beae-bc14a74f6417 service cinder] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:03:36.323946 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1085/2164] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:03:36 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:03:38.928236 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f2e601-9401-4381-9a77-d1649be6452d None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:03:39.005541 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f2e601-9401-4381-9a77-d1649be6452d None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:03:39.013982 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f807b4-5159-46cb-a891-68133d186af1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:03:39.015119 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f807b4-5159-46cb-a891-68133d186af1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:03:39.015563 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f807b4-5159-46cb-a891-68133d186af1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:03:39.015953 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f807b4-5159-46cb-a891-68133d186af1 admin admin] PATH_INFO: `/v3/users/b5daeff9822142978b250c72adcdcfc6` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:03:39.017983 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f807b4-5159-46cb-a891-68133d186af1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5daeff9822142978b250c72adcdcfc6)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:03:39.020879 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f807b4-5159-46cb-a891-68133d186af1 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:03:39.029158 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:03:39.076838 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-91f807b4-5159-46cb-a891-68133d186af1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5daeff9822142978b250c72adcdcfc6'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:03:39.089077 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-91f807b4-5159-46cb-a891-68133d186af1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5daeff9822142978b250c72adcdcfc6'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:03:39.099418 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-91f807b4-5159-46cb-a891-68133d186af1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5daeff9822142978b250c72adcdcfc6'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:03:39.100590 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-91f807b4-5159-46cb-a891-68133d186af1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5daeff9822142978b250c72adcdcfc6'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:03:39.108424 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-91f807b4-5159-46cb-a891-68133d186af1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5daeff9822142978b250c72adcdcfc6'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:03:39.115099 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1080/2165] 10.222.0.59 () {58 vars in 1224 bytes} [Tue Oct 11 12:03:38 2022] DELETE /identity/v3/users/b5daeff9822142978b250c72adcdcfc6 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:03:39.120956 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b460b53d-70d2-4863-b7eb-9823a84011ec None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:03:39.121449 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b460b53d-70d2-4863-b7eb-9823a84011ec None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:03:39.121590 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b460b53d-70d2-4863-b7eb-9823a84011ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:03:39.141303 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-b460b53d-70d2-4863-b7eb-9823a84011ec None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:03:39.142451 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-b460b53d-70d2-4863-b7eb-9823a84011ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:03:39.206924 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1086/2166] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:03:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:03:39.216978 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f807b4-5159-46cb-a891-68133d186af1 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:03:39.218686 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-91f807b4-5159-46cb-a891-68133d186af1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:03:39.289887 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f807b4-5159-46cb-a891-68133d186af1 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:03:39.294163 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb11e95c-37d2-4857-922d-a9b123279eee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e563fe174db4cc19ae6bf8162cc3d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:03:39.294806 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb11e95c-37d2-4857-922d-a9b123279eee service neutron] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:03:39.295078 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb11e95c-37d2-4857-922d-a9b123279eee service neutron] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:03:39.295351 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb11e95c-37d2-4857-922d-a9b123279eee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:03:39.305089 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb11e95c-37d2-4857-922d-a9b123279eee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:03:39.306543 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb11e95c-37d2-4857-922d-a9b123279eee service neutron] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:03:39.316550 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1081/2167] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:03:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 11 12:03:39.596519 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b460b53d-70d2-4863-b7eb-9823a84011ec None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:03:39.598811 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-b460b53d-70d2-4863-b7eb-9823a84011ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:03:39.612989 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b460b53d-70d2-4863-b7eb-9823a84011ec None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:03:39.618249 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08a6acc-5896-4cff-9155-514be03a0a7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:03:39.618965 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08a6acc-5896-4cff-9155-514be03a0a7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:03:39.619241 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08a6acc-5896-4cff-9155-514be03a0a7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:03:39.619515 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08a6acc-5896-4cff-9155-514be03a0a7d admin admin] PATH_INFO: `/v3/projects/6b5bdf17f2924e7a8a3ea07b707404aa` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:03:39.620757 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08a6acc-5896-4cff-9155-514be03a0a7d admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b5bdf17f2924e7a8a3ea07b707404aa)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:03:39.622734 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08a6acc-5896-4cff-9155-514be03a0a7d admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:03:39.680537 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-b08a6acc-5896-4cff-9155-514be03a0a7d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b5bdf17f2924e7a8a3ea07b707404aa', 'request_id': 'req-b08a6acc-5896-4cff-9155-514be03a0a7d'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:03:39.686456 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-b08a6acc-5896-4cff-9155-514be03a0a7d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b5bdf17f2924e7a8a3ea07b707404aa', 'request_id': 'req-b08a6acc-5896-4cff-9155-514be03a0a7d'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:03:39.693025 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-b08a6acc-5896-4cff-9155-514be03a0a7d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b5bdf17f2924e7a8a3ea07b707404aa', 'request_id': 'req-b08a6acc-5896-4cff-9155-514be03a0a7d'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:03:39.707870 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-b08a6acc-5896-4cff-9155-514be03a0a7d admin admin] The token cache is being invalidate because project 6b5bdf17f2924e7a8a3ea07b707404aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81432) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 12:03:39.708083 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-b08a6acc-5896-4cff-9155-514be03a0a7d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:03:39.708745 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-b08a6acc-5896-4cff-9155-514be03a0a7d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:03:39.710263 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1087/2168] 10.222.0.59 () {58 vars in 1233 bytes} [Tue Oct 11 12:03:39 2022] DELETE /identity/v3/projects/6b5bdf17f2924e7a8a3ea07b707404aa => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:03:39.715413 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccb6512-7fed-47ab-95a4-ac03ed6156af None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:03:39.715664 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccb6512-7fed-47ab-95a4-ac03ed6156af None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:03:39.715931 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccb6512-7fed-47ab-95a4-ac03ed6156af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:03:39.736898 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-7ccb6512-7fed-47ab-95a4-ac03ed6156af None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:03:39.738101 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-7ccb6512-7fed-47ab-95a4-ac03ed6156af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:03:39.804594 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1082/2169] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:03:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 12:03:39.816073 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee8b71b-44cd-4c49-b351-bf3683b1223e None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:03:39.816359 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee8b71b-44cd-4c49-b351-bf3683b1223e None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:03:39.816626 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee8b71b-44cd-4c49-b351-bf3683b1223e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:03:39.837585 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-bee8b71b-44cd-4c49-b351-bf3683b1223e None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:03:39.838850 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-bee8b71b-44cd-4c49-b351-bf3683b1223e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:03:39.848720 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1088/2170] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:03:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:03:39.853734 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccb6512-7fed-47ab-95a4-ac03ed6156af None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:03:39.864205 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccb6512-7fed-47ab-95a4-ac03ed6156af None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:03:39.872284 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b720f2-d757-42ba-8c46-688f42d7a7f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:03:39.872913 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b720f2-d757-42ba-8c46-688f42d7a7f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:03:39.873196 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b720f2-d757-42ba-8c46-688f42d7a7f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:03:39.873467 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b720f2-d757-42ba-8c46-688f42d7a7f6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:03:39.874298 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b720f2-d757-42ba-8c46-688f42d7a7f6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 12:03:39.874615 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b720f2-d757-42ba-8c46-688f42d7a7f6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:03:39.876319 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b720f2-d757-42ba-8c46-688f42d7a7f6 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:03:39.888860 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1083/2171] 10.222.0.59 () {58 vars in 1153 bytes} [Tue Oct 11 12:03:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 12:03:39.893795 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee8b71b-44cd-4c49-b351-bf3683b1223e None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:03:39.904191 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee8b71b-44cd-4c49-b351-bf3683b1223e None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:03:39.909139 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3532a8c-ba4e-47f2-a0a0-331f2112a231 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:03:39.909763 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3532a8c-ba4e-47f2-a0a0-331f2112a231 admin admin] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:03:39.910044 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3532a8c-ba4e-47f2-a0a0-331f2112a231 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:03:39.910318 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3532a8c-ba4e-47f2-a0a0-331f2112a231 admin admin] PATH_INFO: `/v3/projects` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:03:39.911000 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3532a8c-ba4e-47f2-a0a0-331f2112a231 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:03:39.912934 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3532a8c-ba4e-47f2-a0a0-331f2112a231 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:03:39.944665 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1089/2172] 10.222.0.59 () {60 vars in 1153 bytes} [Tue Oct 11 12:03:39 2022] POST /identity/v3/projects => generated 371 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:03:39.949198 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b720f2-d757-42ba-8c46-688f42d7a7f6 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:03:39.960486 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b720f2-d757-42ba-8c46-688f42d7a7f6 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:03:39.975906 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:03:40.033462 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592ef979-843e-4161-9b7a-e5779dfeb8be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:03:40.034088 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592ef979-843e-4161-9b7a-e5779dfeb8be admin admin] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:03:40.034367 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592ef979-843e-4161-9b7a-e5779dfeb8be admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:03:40.034670 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592ef979-843e-4161-9b7a-e5779dfeb8be admin admin] PATH_INFO: `/v3/users` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:03:40.035332 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592ef979-843e-4161-9b7a-e5779dfeb8be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:03:40.037247 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592ef979-843e-4161-9b7a-e5779dfeb8be admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:03:40.059289 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1084/2173] 10.222.0.59 () {60 vars in 1144 bytes} [Tue Oct 11 12:03:39 2022] POST /identity/v3/users => generated 286 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:03:40.063702 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3532a8c-ba4e-47f2-a0a0-331f2112a231 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:03:40.074874 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3532a8c-ba4e-47f2-a0a0-331f2112a231 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:03:40.079799 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a26c6f0-a5d3-4eea-b4a8-bd22a6e19d8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:03:40.080399 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a26c6f0-a5d3-4eea-b4a8-bd22a6e19d8e admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:03:40.080680 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a26c6f0-a5d3-4eea-b4a8-bd22a6e19d8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:03:40.080955 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a26c6f0-a5d3-4eea-b4a8-bd22a6e19d8e admin admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:03:40.081624 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a26c6f0-a5d3-4eea-b4a8-bd22a6e19d8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:03:40.083290 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a26c6f0-a5d3-4eea-b4a8-bd22a6e19d8e admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:03:40.094136 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1090/2174] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:03:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:03:40.098204 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592ef979-843e-4161-9b7a-e5779dfeb8be admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:03:40.108785 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592ef979-843e-4161-9b7a-e5779dfeb8be admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:03:40.114749 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ae37f2-02e3-4d56-b13e-c1ee2d0ffefd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:03:40.115367 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ae37f2-02e3-4d56-b13e-c1ee2d0ffefd admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:03:40.115640 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ae37f2-02e3-4d56-b13e-c1ee2d0ffefd admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:03:40.115907 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ae37f2-02e3-4d56-b13e-c1ee2d0ffefd admin admin] PATH_INFO: `/v3/roles` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:03:40.116551 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ae37f2-02e3-4d56-b13e-c1ee2d0ffefd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:03:40.118173 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ae37f2-02e3-4d56-b13e-c1ee2d0ffefd admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:03:40.129778 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1085/2175] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:03:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:03:40.134188 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a26c6f0-a5d3-4eea-b4a8-bd22a6e19d8e admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:03:40.145098 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a26c6f0-a5d3-4eea-b4a8-bd22a6e19d8e admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:03:40.151494 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bcebfc-7a48-42b0-b27c-2e28f7f9f631 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:03:40.152154 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bcebfc-7a48-42b0-b27c-2e28f7f9f631 admin admin] REQUEST_METHOD: `PUT` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:03:40.152437 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bcebfc-7a48-42b0-b27c-2e28f7f9f631 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:03:40.152703 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bcebfc-7a48-42b0-b27c-2e28f7f9f631 admin admin] PATH_INFO: `/v3/projects/2d578c9014e440fa8434a566c3905dee/users/a714b59f8c0b45b9815e175c365a81fc/roles/de2d2bd26fff4748a79990f97a107164` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:03:40.168226 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bcebfc-7a48-42b0-b27c-2e28f7f9f631 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d578c9014e440fa8434a566c3905dee, user_id=a714b59f8c0b45b9815e175c365a81fc, role_id=de2d2bd26fff4748a79990f97a107164)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:03:40.171098 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bcebfc-7a48-42b0-b27c-2e28f7f9f631 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:03:40.186117 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1091/2176] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 12:03:40 2022] PUT /identity/v3/projects/2d578c9014e440fa8434a566c3905dee/users/a714b59f8c0b45b9815e175c365a81fc/roles/de2d2bd26fff4748a79990f97a107164 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:03:40.196555 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df66880d-f888-4f2c-9f40-634bc0701a48 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:03:40.196846 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df66880d-f888-4f2c-9f40-634bc0701a48 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:03:40.197120 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df66880d-f888-4f2c-9f40-634bc0701a48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:03:40.228264 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-df66880d-f888-4f2c-9f40-634bc0701a48 None None] MFA Rules not processed for user `a714b59f8c0b45b9815e175c365a81fc`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:03:40.229398 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-df66880d-f888-4f2c-9f40-634bc0701a48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:03:40.306495 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1086/2177] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:03:40 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:03:40.318812 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bcebfc-7a48-42b0-b27c-2e28f7f9f631 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:03:40.320808 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-e4bcebfc-7a48-42b0-b27c-2e28f7f9f631 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:03:40.331039 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:03:40.381027 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bcebfc-7a48-42b0-b27c-2e28f7f9f631 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:03:40.385104 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3364c7-4ec5-4860-890f-823717225288 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91a00658b79a42bd91a22a2508b5de34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:03:40.385713 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3364c7-4ec5-4860-890f-823717225288 service glance] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:03:40.385993 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3364c7-4ec5-4860-890f-823717225288 service glance] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:03:40.386271 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3364c7-4ec5-4860-890f-823717225288 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:03:40.396239 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3364c7-4ec5-4860-890f-823717225288 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:03:40.397692 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3364c7-4ec5-4860-890f-823717225288 service glance] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:03:40.410747 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1092/2178] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:03:40 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:03:43.089070 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab061c1-6141-44ac-8676-d26bb4f3611b None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:03:43.089701 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab061c1-6141-44ac-8676-d26bb4f3611b None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:03:43.089701 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab061c1-6141-44ac-8676-d26bb4f3611b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:03:43.110728 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-1ab061c1-6141-44ac-8676-d26bb4f3611b None None] MFA Rules not processed for user `41d3680dda7c4cc8aeb6d086efca0775`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:03:43.111943 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-1ab061c1-6141-44ac-8676-d26bb4f3611b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:03:43.176593 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1087/2179] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:03:43 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:03:50.404103 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=23 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:03:53.168871 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=85,keystone:INSERT=3 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:04:18.082027 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3364c7-4ec5-4860-890f-823717225288 service glance] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:04:18.159711 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3364c7-4ec5-4860-890f-823717225288 service glance] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:04:18.164782 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc27a25-9e6d-46ac-9a62-238127f1b912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:04:18.165447 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc27a25-9e6d-46ac-9a62-238127f1b912 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:04:18.165723 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc27a25-9e6d-46ac-9a62-238127f1b912 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:04:18.165996 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc27a25-9e6d-46ac-9a62-238127f1b912 admin admin] PATH_INFO: `/v3/users/a714b59f8c0b45b9815e175c365a81fc` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:04:18.167244 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc27a25-9e6d-46ac-9a62-238127f1b912 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a714b59f8c0b45b9815e175c365a81fc)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:04:18.169330 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc27a25-9e6d-46ac-9a62-238127f1b912 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:04:18.218967 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-edc27a25-9e6d-46ac-9a62-238127f1b912 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a714b59f8c0b45b9815e175c365a81fc'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:04:18.233794 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-edc27a25-9e6d-46ac-9a62-238127f1b912 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a714b59f8c0b45b9815e175c365a81fc'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:04:18.233794 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-edc27a25-9e6d-46ac-9a62-238127f1b912 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a714b59f8c0b45b9815e175c365a81fc'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:04:18.242380 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-edc27a25-9e6d-46ac-9a62-238127f1b912 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a714b59f8c0b45b9815e175c365a81fc'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:04:18.243237 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-edc27a25-9e6d-46ac-9a62-238127f1b912 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a714b59f8c0b45b9815e175c365a81fc'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:04:18.259183 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1093/2180] 10.222.0.59 () {58 vars in 1224 bytes} [Tue Oct 11 12:04:18 2022] DELETE /identity/v3/users/a714b59f8c0b45b9815e175c365a81fc => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:04:18.265200 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c76479-7a0b-4335-87a0-8573ad37ae7c None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:04:18.265807 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c76479-7a0b-4335-87a0-8573ad37ae7c None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:04:18.266242 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c76479-7a0b-4335-87a0-8573ad37ae7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:04:18.300144 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-c2c76479-7a0b-4335-87a0-8573ad37ae7c None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:04:18.302276 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-c2c76479-7a0b-4335-87a0-8573ad37ae7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:04:18.373013 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1088/2181] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:04:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:04:18.383155 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc27a25-9e6d-46ac-9a62-238127f1b912 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:04:18.384665 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-edc27a25-9e6d-46ac-9a62-238127f1b912 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:04:18.449867 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc27a25-9e6d-46ac-9a62-238127f1b912 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:04:18.454227 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7089b978-4265-464f-a458-5272d8536b2a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e563fe174db4cc19ae6bf8162cc3d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:04:18.454896 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7089b978-4265-464f-a458-5272d8536b2a service neutron] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:04:18.455177 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7089b978-4265-464f-a458-5272d8536b2a service neutron] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:04:18.455447 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7089b978-4265-464f-a458-5272d8536b2a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:04:18.466401 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7089b978-4265-464f-a458-5272d8536b2a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:04:18.467891 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7089b978-4265-464f-a458-5272d8536b2a service neutron] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:04:18.478283 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1094/2182] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:04:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:04:18.769168 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c76479-7a0b-4335-87a0-8573ad37ae7c None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:04:18.770911 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-c2c76479-7a0b-4335-87a0-8573ad37ae7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:04:18.785868 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c76479-7a0b-4335-87a0-8573ad37ae7c None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:04:18.791322 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02c6f8f-fa2e-4bad-99bd-65881622e0a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:04:18.794027 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02c6f8f-fa2e-4bad-99bd-65881622e0a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:04:18.794343 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02c6f8f-fa2e-4bad-99bd-65881622e0a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:04:18.794627 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02c6f8f-fa2e-4bad-99bd-65881622e0a2 admin admin] PATH_INFO: `/v3/projects/2d578c9014e440fa8434a566c3905dee` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:04:18.795950 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02c6f8f-fa2e-4bad-99bd-65881622e0a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d578c9014e440fa8434a566c3905dee)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:04:18.797965 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02c6f8f-fa2e-4bad-99bd-65881622e0a2 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:04:18.851512 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-f02c6f8f-fa2e-4bad-99bd-65881622e0a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d578c9014e440fa8434a566c3905dee', 'request_id': 'req-f02c6f8f-fa2e-4bad-99bd-65881622e0a2'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:04:18.858143 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-f02c6f8f-fa2e-4bad-99bd-65881622e0a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d578c9014e440fa8434a566c3905dee', 'request_id': 'req-f02c6f8f-fa2e-4bad-99bd-65881622e0a2'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:04:18.869343 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-f02c6f8f-fa2e-4bad-99bd-65881622e0a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d578c9014e440fa8434a566c3905dee', 'request_id': 'req-f02c6f8f-fa2e-4bad-99bd-65881622e0a2'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:04:18.882584 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-f02c6f8f-fa2e-4bad-99bd-65881622e0a2 admin admin] The token cache is being invalidate because project 2d578c9014e440fa8434a566c3905dee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 12:04:18.882877 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-f02c6f8f-fa2e-4bad-99bd-65881622e0a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:04:18.883598 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-f02c6f8f-fa2e-4bad-99bd-65881622e0a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:04:18.885218 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1089/2183] 10.222.0.59 () {58 vars in 1233 bytes} [Tue Oct 11 12:04:18 2022] DELETE /identity/v3/projects/2d578c9014e440fa8434a566c3905dee => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:04:18.890827 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50740014-65e8-4b0f-9bde-7b91e2e7561f None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:04:18.891286 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50740014-65e8-4b0f-9bde-7b91e2e7561f None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:04:18.891746 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50740014-65e8-4b0f-9bde-7b91e2e7561f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:04:18.915074 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-50740014-65e8-4b0f-9bde-7b91e2e7561f None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:04:18.916311 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-50740014-65e8-4b0f-9bde-7b91e2e7561f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:04:18.937383 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:04:18.987279 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1095/2184] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:04:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:04:18.997154 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ff387b-8026-4932-8afa-d3fc04fc0f8c None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:04:18.997426 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ff387b-8026-4932-8afa-d3fc04fc0f8c None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:04:18.997714 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ff387b-8026-4932-8afa-d3fc04fc0f8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:04:19.018983 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-19ff387b-8026-4932-8afa-d3fc04fc0f8c None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:04:19.020117 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-19ff387b-8026-4932-8afa-d3fc04fc0f8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:04:19.029562 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1090/2185] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:04:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:04:19.034337 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50740014-65e8-4b0f-9bde-7b91e2e7561f None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:04:19.044774 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50740014-65e8-4b0f-9bde-7b91e2e7561f None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:04:19.050347 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71842f8f-09fb-4301-a48f-6e7446772fa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:04:19.051153 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71842f8f-09fb-4301-a48f-6e7446772fa8 admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:04:19.051584 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71842f8f-09fb-4301-a48f-6e7446772fa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:04:19.051862 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71842f8f-09fb-4301-a48f-6e7446772fa8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:04:19.052528 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71842f8f-09fb-4301-a48f-6e7446772fa8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 12:04:19.052843 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71842f8f-09fb-4301-a48f-6e7446772fa8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:04:19.054526 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71842f8f-09fb-4301-a48f-6e7446772fa8 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:04:19.068374 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1096/2186] 10.222.0.59 () {58 vars in 1153 bytes} [Tue Oct 11 12:04:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 12:04:19.073164 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ff387b-8026-4932-8afa-d3fc04fc0f8c None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:04:19.084088 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ff387b-8026-4932-8afa-d3fc04fc0f8c None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:04:19.089106 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c6909f-66a6-492a-80b4-6944551a3b86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:04:19.089747 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c6909f-66a6-492a-80b4-6944551a3b86 admin admin] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:04:19.090029 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c6909f-66a6-492a-80b4-6944551a3b86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:04:19.090303 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c6909f-66a6-492a-80b4-6944551a3b86 admin admin] PATH_INFO: `/v3/projects` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:04:19.090983 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c6909f-66a6-492a-80b4-6944551a3b86 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:04:19.092925 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c6909f-66a6-492a-80b4-6944551a3b86 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:04:19.123404 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1091/2187] 10.222.0.59 () {60 vars in 1153 bytes} [Tue Oct 11 12:04:19 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:04:19.127842 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71842f8f-09fb-4301-a48f-6e7446772fa8 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:04:19.139143 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71842f8f-09fb-4301-a48f-6e7446772fa8 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:04:19.202445 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627f3bab-5aa5-418f-ab5e-e852fc56540d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:04:19.203117 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627f3bab-5aa5-418f-ab5e-e852fc56540d admin admin] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:04:19.203398 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627f3bab-5aa5-418f-ab5e-e852fc56540d admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:04:19.203674 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627f3bab-5aa5-418f-ab5e-e852fc56540d admin admin] PATH_INFO: `/v3/users` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:04:19.204456 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627f3bab-5aa5-418f-ab5e-e852fc56540d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:04:19.206457 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627f3bab-5aa5-418f-ab5e-e852fc56540d admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:04:19.231245 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1097/2188] 10.222.0.59 () {60 vars in 1144 bytes} [Tue Oct 11 12:04:19 2022] POST /identity/v3/users => generated 292 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:04:19.236555 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c6909f-66a6-492a-80b4-6944551a3b86 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:04:19.248624 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c6909f-66a6-492a-80b4-6944551a3b86 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:04:19.254040 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf13b40f-eb44-45a8-a1d1-7465b1dcb15c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:04:19.254750 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf13b40f-eb44-45a8-a1d1-7465b1dcb15c admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:04:19.255013 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf13b40f-eb44-45a8-a1d1-7465b1dcb15c admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:04:19.255286 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf13b40f-eb44-45a8-a1d1-7465b1dcb15c admin admin] PATH_INFO: `/v3/roles` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:04:19.256134 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf13b40f-eb44-45a8-a1d1-7465b1dcb15c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:04:19.257616 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf13b40f-eb44-45a8-a1d1-7465b1dcb15c admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:04:19.268934 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1092/2189] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:04:19 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 11 12:04:19.273733 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627f3bab-5aa5-418f-ab5e-e852fc56540d admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:04:19.284229 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627f3bab-5aa5-418f-ab5e-e852fc56540d admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:04:19.289649 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919722e6-ed78-4024-bb06-fd5e6d2752d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:04:19.290334 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919722e6-ed78-4024-bb06-fd5e6d2752d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:04:19.290577 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919722e6-ed78-4024-bb06-fd5e6d2752d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:04:19.290887 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919722e6-ed78-4024-bb06-fd5e6d2752d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:04:19.291557 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919722e6-ed78-4024-bb06-fd5e6d2752d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:04:19.293203 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919722e6-ed78-4024-bb06-fd5e6d2752d8 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:04:19.305179 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1098/2190] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:04:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:04:19.309722 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf13b40f-eb44-45a8-a1d1-7465b1dcb15c admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:04:19.320586 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf13b40f-eb44-45a8-a1d1-7465b1dcb15c admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:04:19.325818 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4767159-8de4-4b14-8678-37dc401428fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:04:19.326546 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4767159-8de4-4b14-8678-37dc401428fb admin admin] REQUEST_METHOD: `PUT` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:04:19.326851 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4767159-8de4-4b14-8678-37dc401428fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:04:19.327127 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4767159-8de4-4b14-8678-37dc401428fb admin admin] PATH_INFO: `/v3/projects/8908fef147b141b088ec5d458112e624/users/e4e098a260b94b2089801f02bdb0b126/roles/de2d2bd26fff4748a79990f97a107164` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:04:19.345568 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4767159-8de4-4b14-8678-37dc401428fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=8908fef147b141b088ec5d458112e624, user_id=e4e098a260b94b2089801f02bdb0b126, role_id=de2d2bd26fff4748a79990f97a107164)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:04:19.348363 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4767159-8de4-4b14-8678-37dc401428fb admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:04:19.365909 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1093/2191] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 12:04:19 2022] PUT /identity/v3/projects/8908fef147b141b088ec5d458112e624/users/e4e098a260b94b2089801f02bdb0b126/roles/de2d2bd26fff4748a79990f97a107164 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:04:19.371403 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0865c003-ccc9-447e-b7aa-68a1e5875f3c None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:04:19.371665 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0865c003-ccc9-447e-b7aa-68a1e5875f3c None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:04:19.371917 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0865c003-ccc9-447e-b7aa-68a1e5875f3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:04:19.394013 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-0865c003-ccc9-447e-b7aa-68a1e5875f3c None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:04:19.395311 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-0865c003-ccc9-447e-b7aa-68a1e5875f3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:04:19.468771 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1099/2192] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:04:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:04:19.479379 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4767159-8de4-4b14-8678-37dc401428fb admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:04:19.481210 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-c4767159-8de4-4b14-8678-37dc401428fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:04:19.506114 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:04:19.549573 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4767159-8de4-4b14-8678-37dc401428fb admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:04:19.553465 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e142333-4f6c-4f25-b74f-d5442c556332 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e563fe174db4cc19ae6bf8162cc3d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:04:19.554103 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e142333-4f6c-4f25-b74f-d5442c556332 service neutron] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:04:19.554382 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e142333-4f6c-4f25-b74f-d5442c556332 service neutron] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:04:19.554694 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e142333-4f6c-4f25-b74f-d5442c556332 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:04:19.564863 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e142333-4f6c-4f25-b74f-d5442c556332 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:04:19.566325 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e142333-4f6c-4f25-b74f-d5442c556332 service neutron] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:04:19.577165 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1094/2193] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:04:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:04:20.250380 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8cfce1-be03-4785-9f98-228238c8fa90 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:04:20.251095 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8cfce1-be03-4785-9f98-228238c8fa90 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:04:20.251095 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8cfce1-be03-4785-9f98-228238c8fa90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:04:20.283116 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-bb8cfce1-be03-4785-9f98-228238c8fa90 None None] MFA Rules not processed for user `e4e098a260b94b2089801f02bdb0b126`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:04:20.284370 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-bb8cfce1-be03-4785-9f98-228238c8fa90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:04:20.303971 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:04:20.361926 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1100/2194] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:04:20 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:04:20.373782 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e142333-4f6c-4f25-b74f-d5442c556332 service neutron] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:04:20.375526 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-3e142333-4f6c-4f25-b74f-d5442c556332 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:04:20.444632 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e142333-4f6c-4f25-b74f-d5442c556332 service neutron] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:04:20.448378 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b37d9f-da55-4f5c-8e9b-2f0e0263112f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b1b651c73664d08accc6edf2f79bace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:04:20.448999 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b37d9f-da55-4f5c-8e9b-2f0e0263112f service cinder] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:04:20.449274 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b37d9f-da55-4f5c-8e9b-2f0e0263112f service cinder] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:04:20.449543 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b37d9f-da55-4f5c-8e9b-2f0e0263112f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:04:20.459377 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b37d9f-da55-4f5c-8e9b-2f0e0263112f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:04:20.461233 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b37d9f-da55-4f5c-8e9b-2f0e0263112f service cinder] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:04:20.472414 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1095/2195] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:04:20 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:04:21.922223 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14ef946-e194-428b-a35d-849e604c8916 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:04:21.922873 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14ef946-e194-428b-a35d-849e604c8916 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:04:21.922873 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14ef946-e194-428b-a35d-849e604c8916 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:04:21.943312 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-f14ef946-e194-428b-a35d-849e604c8916 None None] MFA Rules not processed for user `e4e098a260b94b2089801f02bdb0b126`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:04:21.944599 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-f14ef946-e194-428b-a35d-849e604c8916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:04:21.951994 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1101/2196] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:04:21 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:04:30.467065 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=43 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:04:31.937728 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=22 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:04:54.878070 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b37d9f-da55-4f5c-8e9b-2f0e0263112f service cinder] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:04:54.879586 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-01b37d9f-da55-4f5c-8e9b-2f0e0263112f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:04:54.891621 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b37d9f-da55-4f5c-8e9b-2f0e0263112f service cinder] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:04:54.895848 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814981ff-811c-4a36-ab00-a3c00c21e73a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b1b651c73664d08accc6edf2f79bace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:04:54.896504 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814981ff-811c-4a36-ab00-a3c00c21e73a service cinder] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:04:54.896821 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814981ff-811c-4a36-ab00-a3c00c21e73a service cinder] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:04:54.897095 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814981ff-811c-4a36-ab00-a3c00c21e73a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:04:54.900263 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-814981ff-811c-4a36-ab00-a3c00c21e73a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:04:54.992522 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814981ff-811c-4a36-ab00-a3c00c21e73a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:04:54.994150 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814981ff-811c-4a36-ab00-a3c00c21e73a service cinder] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:04:55.006125 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1096/2197] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:04:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:04:57.468433 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14ef946-e194-428b-a35d-849e604c8916 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:04:57.469875 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-f14ef946-e194-428b-a35d-849e604c8916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:04:57.483572 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14ef946-e194-428b-a35d-849e604c8916 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:04:57.487736 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3384a6-f4b9-4c70-862d-480cc045bdc6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e563fe174db4cc19ae6bf8162cc3d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:04:57.488354 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3384a6-f4b9-4c70-862d-480cc045bdc6 service neutron] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:04:57.488629 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3384a6-f4b9-4c70-862d-480cc045bdc6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:04:57.488896 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3384a6-f4b9-4c70-862d-480cc045bdc6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:04:57.490942 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-ae3384a6-f4b9-4c70-862d-480cc045bdc6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:04:57.505324 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3384a6-f4b9-4c70-862d-480cc045bdc6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:04:57.506840 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3384a6-f4b9-4c70-862d-480cc045bdc6 service neutron] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:04:57.519042 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1102/2198] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:04:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:05:05.000687 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=34 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:05:07.513992 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=6 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:05:10.347760 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814981ff-811c-4a36-ab00-a3c00c21e73a service cinder] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:05:10.349500 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-814981ff-811c-4a36-ab00-a3c00c21e73a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:05:10.364399 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814981ff-811c-4a36-ab00-a3c00c21e73a service cinder] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:05:10.371168 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3607a5d-d42a-422c-a5bd-9a35116f138a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bad5e1cb306b46818096940acc3810b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:05:10.371789 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3607a5d-d42a-422c-a5bd-9a35116f138a service nova] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:05:10.372067 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3607a5d-d42a-422c-a5bd-9a35116f138a service nova] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:05:10.372337 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3607a5d-d42a-422c-a5bd-9a35116f138a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:05:10.374198 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-c3607a5d-d42a-422c-a5bd-9a35116f138a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:05:10.439787 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3607a5d-d42a-422c-a5bd-9a35116f138a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:05:10.441384 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3607a5d-d42a-422c-a5bd-9a35116f138a service nova] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:05:10.451657 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1097/2199] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:05:10 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:05:20.446834 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=28 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:05:30.844133 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d32d67-49d7-409c-a6c2-d78b9deb3bf0 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:05:30.844762 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d32d67-49d7-409c-a6c2-d78b9deb3bf0 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:05:30.844860 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d32d67-49d7-409c-a6c2-d78b9deb3bf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:05:30.864957 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-e4d32d67-49d7-409c-a6c2-d78b9deb3bf0 None None] MFA Rules not processed for user `c58107eee1b94541a330c26d7b82a53b`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:05:30.866206 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-e4d32d67-49d7-409c-a6c2-d78b9deb3bf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:05:30.873408 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1103/2200] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:05:30 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:05:31.227777 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3607a5d-d42a-422c-a5bd-9a35116f138a service nova] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:05:31.240755 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3607a5d-d42a-422c-a5bd-9a35116f138a service nova] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:05:31.247427 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8d1879-37cd-4c97-963b-2f04be54f846 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e563fe174db4cc19ae6bf8162cc3d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:05:31.248092 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8d1879-37cd-4c97-963b-2f04be54f846 service neutron] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:05:31.248377 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8d1879-37cd-4c97-963b-2f04be54f846 service neutron] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:05:31.248654 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8d1879-37cd-4c97-963b-2f04be54f846 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:05:31.250701 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-bd8d1879-37cd-4c97-963b-2f04be54f846 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:05:31.267860 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8d1879-37cd-4c97-963b-2f04be54f846 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:05:31.269881 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8d1879-37cd-4c97-963b-2f04be54f846 service neutron] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:05:31.280539 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1098/2201] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:05:31 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:05:37.293153 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d32d67-49d7-409c-a6c2-d78b9deb3bf0 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:05:37.294668 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-e4d32d67-49d7-409c-a6c2-d78b9deb3bf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:05:37.311614 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d32d67-49d7-409c-a6c2-d78b9deb3bf0 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:05:37.318519 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876e7209-97df-418a-946d-8dd8e7262a36 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bad5e1cb306b46818096940acc3810b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:05:37.319191 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876e7209-97df-418a-946d-8dd8e7262a36 service nova] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:05:37.319475 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876e7209-97df-418a-946d-8dd8e7262a36 service nova] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:05:37.319747 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876e7209-97df-418a-946d-8dd8e7262a36 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:05:37.321983 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-876e7209-97df-418a-946d-8dd8e7262a36 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:05:37.334582 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876e7209-97df-418a-946d-8dd8e7262a36 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:05:37.336275 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876e7209-97df-418a-946d-8dd8e7262a36 service nova] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:05:37.348507 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1104/2202] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:05:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:05:41.275621 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=6 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:05:47.343283 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=9 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:05:49.498217 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500a555a-3cd4-4617-b4d1-ae50665957eb None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:05:49.498783 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500a555a-3cd4-4617-b4d1-ae50665957eb None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:05:49.498783 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500a555a-3cd4-4617-b4d1-ae50665957eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:05:49.520273 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-500a555a-3cd4-4617-b4d1-ae50665957eb None None] MFA Rules not processed for user `e4e098a260b94b2089801f02bdb0b126`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:05:49.521820 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-500a555a-3cd4-4617-b4d1-ae50665957eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:05:49.529476 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1099/2203] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:05:49 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:05:49.796616 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876e7209-97df-418a-946d-8dd8e7262a36 service nova] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:05:49.810006 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876e7209-97df-418a-946d-8dd8e7262a36 service nova] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:05:49.816576 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97f440a-cf5d-4008-80ee-231d38cc4cb8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e563fe174db4cc19ae6bf8162cc3d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:05:49.817398 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97f440a-cf5d-4008-80ee-231d38cc4cb8 service neutron] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:05:49.818521 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97f440a-cf5d-4008-80ee-231d38cc4cb8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:05:49.818835 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97f440a-cf5d-4008-80ee-231d38cc4cb8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:05:49.822152 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-b97f440a-cf5d-4008-80ee-231d38cc4cb8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:05:49.834826 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97f440a-cf5d-4008-80ee-231d38cc4cb8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:05:49.836325 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97f440a-cf5d-4008-80ee-231d38cc4cb8 service neutron] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:05:49.845215 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1105/2204] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:05:49 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:05:50.194044 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500a555a-3cd4-4617-b4d1-ae50665957eb None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:05:50.195811 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-500a555a-3cd4-4617-b4d1-ae50665957eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:05:50.280154 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500a555a-3cd4-4617-b4d1-ae50665957eb None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:05:50.286517 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0083ac67-bf53-4294-93f2-2db476806fae service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f272cdd1bcc04b4e8261ed533cfd4fd7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:05:50.287165 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0083ac67-bf53-4294-93f2-2db476806fae service placement] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:05:50.288758 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0083ac67-bf53-4294-93f2-2db476806fae service placement] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:05:50.288758 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0083ac67-bf53-4294-93f2-2db476806fae service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:05:50.289839 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-0083ac67-bf53-4294-93f2-2db476806fae service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:05:50.302544 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0083ac67-bf53-4294-93f2-2db476806fae service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:05:50.303951 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0083ac67-bf53-4294-93f2-2db476806fae service placement] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:05:50.314432 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1100/2205] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:05:50 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:05:52.681413 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97f440a-cf5d-4008-80ee-231d38cc4cb8 service neutron] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:05:52.697717 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97f440a-cf5d-4008-80ee-231d38cc4cb8 service neutron] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:05:52.704839 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e749d1-0dd6-467a-8eea-72a42817be8e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bad5e1cb306b46818096940acc3810b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:05:52.705469 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e749d1-0dd6-467a-8eea-72a42817be8e service nova] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:05:52.705783 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e749d1-0dd6-467a-8eea-72a42817be8e service nova] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:05:52.706062 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e749d1-0dd6-467a-8eea-72a42817be8e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:05:52.708841 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-00e749d1-0dd6-467a-8eea-72a42817be8e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:05:52.794883 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e749d1-0dd6-467a-8eea-72a42817be8e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:05:52.797915 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e749d1-0dd6-467a-8eea-72a42817be8e service nova] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:05:52.816122 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1106/2206] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:05:52 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:06:00.309986 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=35 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:06:02.808698 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=34 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:06:02.910032 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0083ac67-bf53-4294-93f2-2db476806fae service placement] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:06:02.935876 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0083ac67-bf53-4294-93f2-2db476806fae service placement] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:06:02.946979 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115ed2fb-323e-4f17-a0d6-12f9d486e402 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bad5e1cb306b46818096940acc3810b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:06:02.948053 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115ed2fb-323e-4f17-a0d6-12f9d486e402 service nova] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:06:02.948556 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115ed2fb-323e-4f17-a0d6-12f9d486e402 service nova] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:06:02.949062 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115ed2fb-323e-4f17-a0d6-12f9d486e402 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:06:02.952098 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-115ed2fb-323e-4f17-a0d6-12f9d486e402 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:06:02.984027 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115ed2fb-323e-4f17-a0d6-12f9d486e402 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:06:02.988920 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115ed2fb-323e-4f17-a0d6-12f9d486e402 service nova] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:06:03.010124 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1101/2207] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:06:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:06:13.001333 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=6 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:06:13.939429 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2300f6-36f8-4f72-b578-c2064c503b6b None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:06:13.940006 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2300f6-36f8-4f72-b578-c2064c503b6b None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:06:13.940006 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2300f6-36f8-4f72-b578-c2064c503b6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:06:13.963153 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-8b2300f6-36f8-4f72-b578-c2064c503b6b None None] MFA Rules not processed for user `41d3680dda7c4cc8aeb6d086efca0775`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:06:13.964499 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-8b2300f6-36f8-4f72-b578-c2064c503b6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:06:13.976954 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1107/2208] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:06:13 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:06:14.306856 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115ed2fb-323e-4f17-a0d6-12f9d486e402 service nova] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:06:14.317375 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115ed2fb-323e-4f17-a0d6-12f9d486e402 service nova] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:06:14.321612 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f597de-473c-476d-9ba7-ed3715168f9f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e563fe174db4cc19ae6bf8162cc3d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:06:14.322284 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f597de-473c-476d-9ba7-ed3715168f9f service neutron] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:06:14.322580 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f597de-473c-476d-9ba7-ed3715168f9f service neutron] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:06:14.322907 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f597de-473c-476d-9ba7-ed3715168f9f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:06:14.324959 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-25f597de-473c-476d-9ba7-ed3715168f9f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:06:14.335671 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f597de-473c-476d-9ba7-ed3715168f9f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:06:14.337107 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f597de-473c-476d-9ba7-ed3715168f9f service neutron] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:06:14.345877 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1102/2209] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:06:14 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:06:23.956449 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=3 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:06:24.342000 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=6 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:06:57.541677 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2300f6-36f8-4f72-b578-c2064c503b6b None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:06:57.556750 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2300f6-36f8-4f72-b578-c2064c503b6b None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:06:57.564128 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9461ed-5acb-47c0-ae72-66797e2be164 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f272cdd1bcc04b4e8261ed533cfd4fd7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:06:57.565205 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9461ed-5acb-47c0-ae72-66797e2be164 service placement] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:06:57.565693 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9461ed-5acb-47c0-ae72-66797e2be164 service placement] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:06:57.566123 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9461ed-5acb-47c0-ae72-66797e2be164 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:06:57.569432 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-3b9461ed-5acb-47c0-ae72-66797e2be164 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:06:57.589450 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9461ed-5acb-47c0-ae72-66797e2be164 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:06:57.591767 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9461ed-5acb-47c0-ae72-66797e2be164 service placement] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:06:57.605491 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f597de-473c-476d-9ba7-ed3715168f9f service neutron] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:06:57.607209 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1108/2210] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:06:57 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:06:57.617760 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f597de-473c-476d-9ba7-ed3715168f9f service neutron] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:06:57.635880 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbafadd0-4a3e-43c4-95af-c20b18fbbcb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:06:57.640698 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbafadd0-4a3e-43c4-95af-c20b18fbbcb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:06:57.641197 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbafadd0-4a3e-43c4-95af-c20b18fbbcb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:06:57.641459 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbafadd0-4a3e-43c4-95af-c20b18fbbcb4 admin admin] PATH_INFO: `/v3/users/e4e098a260b94b2089801f02bdb0b126` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:06:57.642938 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbafadd0-4a3e-43c4-95af-c20b18fbbcb4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4e098a260b94b2089801f02bdb0b126)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:06:57.645229 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbafadd0-4a3e-43c4-95af-c20b18fbbcb4 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:06:57.701781 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-dbafadd0-4a3e-43c4-95af-c20b18fbbcb4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4e098a260b94b2089801f02bdb0b126'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:06:57.714169 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-dbafadd0-4a3e-43c4-95af-c20b18fbbcb4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4e098a260b94b2089801f02bdb0b126'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:06:57.723145 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-dbafadd0-4a3e-43c4-95af-c20b18fbbcb4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4e098a260b94b2089801f02bdb0b126'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:06:57.723803 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-dbafadd0-4a3e-43c4-95af-c20b18fbbcb4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4e098a260b94b2089801f02bdb0b126'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:06:57.732253 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-dbafadd0-4a3e-43c4-95af-c20b18fbbcb4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4e098a260b94b2089801f02bdb0b126'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:06:57.740912 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1103/2211] 10.222.0.59 () {58 vars in 1224 bytes} [Tue Oct 11 12:06:57 2022] DELETE /identity/v3/users/e4e098a260b94b2089801f02bdb0b126 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 12:06:57.939042 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9461ed-5acb-47c0-ae72-66797e2be164 service placement] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:06:57.952073 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9461ed-5acb-47c0-ae72-66797e2be164 service placement] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:06:58.019214 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad7968a-dd2a-4ac2-b3f7-8f8aebc989cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:06:58.020082 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad7968a-dd2a-4ac2-b3f7-8f8aebc989cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:06:58.020361 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad7968a-dd2a-4ac2-b3f7-8f8aebc989cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:06:58.020634 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad7968a-dd2a-4ac2-b3f7-8f8aebc989cc admin admin] PATH_INFO: `/v3/projects/8908fef147b141b088ec5d458112e624` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:06:58.021844 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad7968a-dd2a-4ac2-b3f7-8f8aebc989cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=8908fef147b141b088ec5d458112e624)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:06:58.023812 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad7968a-dd2a-4ac2-b3f7-8f8aebc989cc admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:06:58.085770 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-4ad7968a-dd2a-4ac2-b3f7-8f8aebc989cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8908fef147b141b088ec5d458112e624', 'request_id': 'req-4ad7968a-dd2a-4ac2-b3f7-8f8aebc989cc'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:06:58.092958 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-4ad7968a-dd2a-4ac2-b3f7-8f8aebc989cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8908fef147b141b088ec5d458112e624', 'request_id': 'req-4ad7968a-dd2a-4ac2-b3f7-8f8aebc989cc'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:06:58.100002 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-4ad7968a-dd2a-4ac2-b3f7-8f8aebc989cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8908fef147b141b088ec5d458112e624', 'request_id': 'req-4ad7968a-dd2a-4ac2-b3f7-8f8aebc989cc'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:06:58.113959 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-4ad7968a-dd2a-4ac2-b3f7-8f8aebc989cc admin admin] The token cache is being invalidate because project 8908fef147b141b088ec5d458112e624 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81432) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 12:06:58.114258 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-4ad7968a-dd2a-4ac2-b3f7-8f8aebc989cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:06:58.115044 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-4ad7968a-dd2a-4ac2-b3f7-8f8aebc989cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:06:58.116766 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1109/2212] 10.222.0.59 () {58 vars in 1233 bytes} [Tue Oct 11 12:06:57 2022] DELETE /identity/v3/projects/8908fef147b141b088ec5d458112e624 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:06:58.123565 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8b8fa5-807b-4e0a-9c63-d97131c859be None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:06:58.123887 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8b8fa5-807b-4e0a-9c63-d97131c859be None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:06:58.124146 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8b8fa5-807b-4e0a-9c63-d97131c859be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:06:58.150323 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-3d8b8fa5-807b-4e0a-9c63-d97131c859be None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:06:58.151725 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-3d8b8fa5-807b-4e0a-9c63-d97131c859be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:06:58.221742 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1104/2213] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:06:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:06:58.230491 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f93e2b9-1f53-421e-9537-366fea4da0b4 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:06:58.230845 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f93e2b9-1f53-421e-9537-366fea4da0b4 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:06:58.231121 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f93e2b9-1f53-421e-9537-366fea4da0b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:06:58.251252 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-0f93e2b9-1f53-421e-9537-366fea4da0b4 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:06:58.252493 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-0f93e2b9-1f53-421e-9537-366fea4da0b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:06:58.261752 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1110/2214] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:06:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:06:58.268593 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8b8fa5-807b-4e0a-9c63-d97131c859be None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:06:58.286305 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8b8fa5-807b-4e0a-9c63-d97131c859be None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:06:58.292043 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc40c45-fa19-488f-a876-bbfccd60660e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:06:58.292911 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc40c45-fa19-488f-a876-bbfccd60660e admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:06:58.293384 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc40c45-fa19-488f-a876-bbfccd60660e admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:06:58.293846 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc40c45-fa19-488f-a876-bbfccd60660e admin admin] PATH_INFO: `/v3/domains` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:06:58.294994 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc40c45-fa19-488f-a876-bbfccd60660e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 12:06:58.295534 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc40c45-fa19-488f-a876-bbfccd60660e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:06:58.298601 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc40c45-fa19-488f-a876-bbfccd60660e admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:06:58.313871 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1105/2215] 10.222.0.59 () {58 vars in 1153 bytes} [Tue Oct 11 12:06:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 12:06:58.318772 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f93e2b9-1f53-421e-9537-366fea4da0b4 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:06:58.328348 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f93e2b9-1f53-421e-9537-366fea4da0b4 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:06:58.336975 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4cdc8f-78d7-45ac-85b4-646bc9381dab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:06:58.337612 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4cdc8f-78d7-45ac-85b4-646bc9381dab admin admin] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:06:58.337889 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4cdc8f-78d7-45ac-85b4-646bc9381dab admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:06:58.338146 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4cdc8f-78d7-45ac-85b4-646bc9381dab admin admin] PATH_INFO: `/v3/projects` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:06:58.338851 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4cdc8f-78d7-45ac-85b4-646bc9381dab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:06:58.340870 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4cdc8f-78d7-45ac-85b4-646bc9381dab admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:06:58.373062 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1111/2216] 10.222.0.59 () {60 vars in 1153 bytes} [Tue Oct 11 12:06:58 2022] POST /identity/v3/projects => generated 373 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:06:58.377807 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc40c45-fa19-488f-a876-bbfccd60660e admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:06:58.388961 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc40c45-fa19-488f-a876-bbfccd60660e admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:06:58.457808 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8092a2d3-552c-47f8-8cdd-1adc2ea33cc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:06:58.458429 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8092a2d3-552c-47f8-8cdd-1adc2ea33cc0 admin admin] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:06:58.458743 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8092a2d3-552c-47f8-8cdd-1adc2ea33cc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:06:58.459064 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8092a2d3-552c-47f8-8cdd-1adc2ea33cc0 admin admin] PATH_INFO: `/v3/users` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:06:58.459716 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8092a2d3-552c-47f8-8cdd-1adc2ea33cc0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:06:58.461715 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8092a2d3-552c-47f8-8cdd-1adc2ea33cc0 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:06:58.472688 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:06:58.492536 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1106/2217] 10.222.0.59 () {60 vars in 1144 bytes} [Tue Oct 11 12:06:58 2022] POST /identity/v3/users => generated 287 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:06:58.497623 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4cdc8f-78d7-45ac-85b4-646bc9381dab admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:06:58.509258 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4cdc8f-78d7-45ac-85b4-646bc9381dab admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:06:58.514692 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1705be0-9f37-407c-a0b5-4f1529e3b550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:06:58.515295 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1705be0-9f37-407c-a0b5-4f1529e3b550 admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:06:58.515575 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1705be0-9f37-407c-a0b5-4f1529e3b550 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:06:58.515862 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1705be0-9f37-407c-a0b5-4f1529e3b550 admin admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:06:58.516503 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1705be0-9f37-407c-a0b5-4f1529e3b550 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:06:58.518129 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1705be0-9f37-407c-a0b5-4f1529e3b550 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:06:58.529736 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1112/2218] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:06:58 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:06:58.534895 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8092a2d3-552c-47f8-8cdd-1adc2ea33cc0 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:06:58.546049 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8092a2d3-552c-47f8-8cdd-1adc2ea33cc0 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:06:58.551697 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ad34e8-c1a5-43bd-aaf0-017b1197ffa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:06:58.552306 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ad34e8-c1a5-43bd-aaf0-017b1197ffa4 admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:06:58.552579 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ad34e8-c1a5-43bd-aaf0-017b1197ffa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:06:58.552914 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ad34e8-c1a5-43bd-aaf0-017b1197ffa4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:06:58.553632 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ad34e8-c1a5-43bd-aaf0-017b1197ffa4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:06:58.555303 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ad34e8-c1a5-43bd-aaf0-017b1197ffa4 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:06:58.567176 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1107/2219] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:06:58 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:06:58.571784 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1705be0-9f37-407c-a0b5-4f1529e3b550 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:06:58.586658 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1705be0-9f37-407c-a0b5-4f1529e3b550 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:06:58.594808 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b9dbe5-10c2-4158-9f15-d78a5885cac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:06:58.595937 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b9dbe5-10c2-4158-9f15-d78a5885cac2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:06:58.596445 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b9dbe5-10c2-4158-9f15-d78a5885cac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:06:58.596925 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b9dbe5-10c2-4158-9f15-d78a5885cac2 admin admin] PATH_INFO: `/v3/projects/ef7f8c1858da4f73925e63ea06100eab/users/edb722b67a804915bf38f34b67845bc9/roles/de2d2bd26fff4748a79990f97a107164` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:06:58.623916 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b9dbe5-10c2-4158-9f15-d78a5885cac2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef7f8c1858da4f73925e63ea06100eab, user_id=edb722b67a804915bf38f34b67845bc9, role_id=de2d2bd26fff4748a79990f97a107164)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:06:58.629453 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b9dbe5-10c2-4158-9f15-d78a5885cac2 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:06:58.651941 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1113/2220] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 12:06:58 2022] PUT /identity/v3/projects/ef7f8c1858da4f73925e63ea06100eab/users/edb722b67a804915bf38f34b67845bc9/roles/de2d2bd26fff4748a79990f97a107164 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:06:58.657182 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a372d151-8678-4ed8-bedf-ea0bdcc29e61 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:06:58.657987 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a372d151-8678-4ed8-bedf-ea0bdcc29e61 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:06:58.657987 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a372d151-8678-4ed8-bedf-ea0bdcc29e61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:06:58.679666 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-a372d151-8678-4ed8-bedf-ea0bdcc29e61 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:06:58.680905 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-a372d151-8678-4ed8-bedf-ea0bdcc29e61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:06:58.757038 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1108/2221] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:06:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:06:58.769013 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b9dbe5-10c2-4158-9f15-d78a5885cac2 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:06:58.771629 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-04b9dbe5-10c2-4158-9f15-d78a5885cac2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:06:58.790964 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:06:58.852217 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b9dbe5-10c2-4158-9f15-d78a5885cac2 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:06:58.856607 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ed7a80-45f4-42c2-bef2-2da81e5e5a5d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e563fe174db4cc19ae6bf8162cc3d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:06:58.857318 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ed7a80-45f4-42c2-bef2-2da81e5e5a5d service neutron] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:06:58.857598 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ed7a80-45f4-42c2-bef2-2da81e5e5a5d service neutron] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:06:58.857880 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ed7a80-45f4-42c2-bef2-2da81e5e5a5d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:06:58.870524 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ed7a80-45f4-42c2-bef2-2da81e5e5a5d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:06:58.872095 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ed7a80-45f4-42c2-bef2-2da81e5e5a5d service neutron] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:06:58.884443 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1114/2222] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:06:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:07:04.186328 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38058217-45a2-48b6-88b2-d6fc75ed23b5 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:07:04.187298 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38058217-45a2-48b6-88b2-d6fc75ed23b5 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:07:04.188034 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38058217-45a2-48b6-88b2-d6fc75ed23b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:07:04.235528 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-38058217-45a2-48b6-88b2-d6fc75ed23b5 None None] MFA Rules not processed for user `edb722b67a804915bf38f34b67845bc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:07:04.238261 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-38058217-45a2-48b6-88b2-d6fc75ed23b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:07:04.363821 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1109/2223] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:07:04 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:07:04.368468 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ed7a80-45f4-42c2-bef2-2da81e5e5a5d service neutron] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:07:04.382245 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ed7a80-45f4-42c2-bef2-2da81e5e5a5d service neutron] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:07:04.395217 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c07fd9b-dde7-4b4d-adf5-96308f83d4ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:07:04.395887 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c07fd9b-dde7-4b4d-adf5-96308f83d4ba admin admin] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:07:04.396215 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c07fd9b-dde7-4b4d-adf5-96308f83d4ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:07:04.396496 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c07fd9b-dde7-4b4d-adf5-96308f83d4ba admin admin] PATH_INFO: `/v3/projects` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:07:04.397155 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c07fd9b-dde7-4b4d-adf5-96308f83d4ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:07:04.399173 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c07fd9b-dde7-4b4d-adf5-96308f83d4ba admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:07:04.436426 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1115/2224] 10.222.0.59 () {60 vars in 1153 bytes} [Tue Oct 11 12:07:04 2022] POST /identity/v3/projects => generated 371 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:07:04.442399 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38058217-45a2-48b6-88b2-d6fc75ed23b5 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:07:04.462551 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38058217-45a2-48b6-88b2-d6fc75ed23b5 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:07:04.575438 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e02501-aab9-4d3c-bdad-963daedc8c78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:07:04.576131 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e02501-aab9-4d3c-bdad-963daedc8c78 admin admin] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:07:04.576613 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e02501-aab9-4d3c-bdad-963daedc8c78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:07:04.576904 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e02501-aab9-4d3c-bdad-963daedc8c78 admin admin] PATH_INFO: `/v3/users` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:07:04.577578 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e02501-aab9-4d3c-bdad-963daedc8c78 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:07:04.581674 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e02501-aab9-4d3c-bdad-963daedc8c78 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:07:04.609466 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:07:04.616179 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1110/2225] 10.222.0.59 () {60 vars in 1144 bytes} [Tue Oct 11 12:07:04 2022] POST /identity/v3/users => generated 292 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:07:04.620197 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c07fd9b-dde7-4b4d-adf5-96308f83d4ba admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:07:04.634372 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c07fd9b-dde7-4b4d-adf5-96308f83d4ba admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:07:04.646831 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fbb1d4-76f9-4b71-8c8a-a3476c337d9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:07:04.647846 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fbb1d4-76f9-4b71-8c8a-a3476c337d9b admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:07:04.648331 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fbb1d4-76f9-4b71-8c8a-a3476c337d9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:07:04.648698 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fbb1d4-76f9-4b71-8c8a-a3476c337d9b admin admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:07:04.650027 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fbb1d4-76f9-4b71-8c8a-a3476c337d9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:07:04.654819 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fbb1d4-76f9-4b71-8c8a-a3476c337d9b admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:07:04.682043 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1116/2226] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:07:04 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 11 12:07:04.687212 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e02501-aab9-4d3c-bdad-963daedc8c78 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:07:04.707325 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e02501-aab9-4d3c-bdad-963daedc8c78 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:07:04.719338 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643e9c39-5d1f-48e3-b93f-323ab0325ac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:07:04.720222 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643e9c39-5d1f-48e3-b93f-323ab0325ac8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:07:04.720509 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643e9c39-5d1f-48e3-b93f-323ab0325ac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:07:04.720785 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643e9c39-5d1f-48e3-b93f-323ab0325ac8 admin admin] PATH_INFO: `/v3/projects/5b8a4b5ccf924d7da68aec4a0fbf1281/users/15bc8f7341dd46bb83584f5dc4d5d50d/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:07:04.744453 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643e9c39-5d1f-48e3-b93f-323ab0325ac8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b8a4b5ccf924d7da68aec4a0fbf1281, user_id=15bc8f7341dd46bb83584f5dc4d5d50d, role_id=5cf1f4bd2fef46ab9c6eff56d52f9f69)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:07:04.747358 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643e9c39-5d1f-48e3-b93f-323ab0325ac8 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:07:04.774462 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1111/2227] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 12:07:04 2022] PUT /identity/v3/projects/5b8a4b5ccf924d7da68aec4a0fbf1281/users/15bc8f7341dd46bb83584f5dc4d5d50d/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:07:04.776434 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fbb1d4-76f9-4b71-8c8a-a3476c337d9b admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:07:04.791692 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fbb1d4-76f9-4b71-8c8a-a3476c337d9b admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:07:04.883120 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2657ab2b-72fc-4627-9344-ec21c2d2331f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:07:04.883912 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2657ab2b-72fc-4627-9344-ec21c2d2331f admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:07:04.884199 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2657ab2b-72fc-4627-9344-ec21c2d2331f admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:07:04.884479 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2657ab2b-72fc-4627-9344-ec21c2d2331f admin admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:07:04.885139 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2657ab2b-72fc-4627-9344-ec21c2d2331f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:07:04.886889 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2657ab2b-72fc-4627-9344-ec21c2d2331f admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:07:04.899863 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1117/2228] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:07:04 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:07:04.906128 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643e9c39-5d1f-48e3-b93f-323ab0325ac8 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:07:04.920723 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643e9c39-5d1f-48e3-b93f-323ab0325ac8 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:07:04.926077 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721f2104-e780-4eb2-ab58-e56bdb22d8c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:07:04.926813 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721f2104-e780-4eb2-ab58-e56bdb22d8c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:07:04.927094 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721f2104-e780-4eb2-ab58-e56bdb22d8c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:07:04.927410 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721f2104-e780-4eb2-ab58-e56bdb22d8c1 admin admin] PATH_INFO: `/v3/projects/5b8a4b5ccf924d7da68aec4a0fbf1281/users/15bc8f7341dd46bb83584f5dc4d5d50d/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:07:04.929522 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721f2104-e780-4eb2-ab58-e56bdb22d8c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b8a4b5ccf924d7da68aec4a0fbf1281, user_id=15bc8f7341dd46bb83584f5dc4d5d50d, role_id=5cf1f4bd2fef46ab9c6eff56d52f9f69)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:07:04.932422 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721f2104-e780-4eb2-ab58-e56bdb22d8c1 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:07:04.947241 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1112/2229] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 12:07:04 2022] PUT /identity/v3/projects/5b8a4b5ccf924d7da68aec4a0fbf1281/users/15bc8f7341dd46bb83584f5dc4d5d50d/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 12:07:05.931974 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2657ab2b-72fc-4627-9344-ec21c2d2331f admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:07:05.933973 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-2657ab2b-72fc-4627-9344-ec21c2d2331f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:07:06.049314 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2657ab2b-72fc-4627-9344-ec21c2d2331f admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:07:06.072884 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:07:06.157831 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8264345e-c4d1-414a-9265-759d4677f29a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bad5e1cb306b46818096940acc3810b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:07:06.159087 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8264345e-c4d1-414a-9265-759d4677f29a service nova] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:07:06.159725 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8264345e-c4d1-414a-9265-759d4677f29a service nova] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:07:06.160302 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8264345e-c4d1-414a-9265-759d4677f29a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:07:06.164021 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-8264345e-c4d1-414a-9265-759d4677f29a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:07:06.252217 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8264345e-c4d1-414a-9265-759d4677f29a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:07:06.254442 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8264345e-c4d1-414a-9265-759d4677f29a service nova] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:07:06.274797 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1118/2230] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:07:05 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 11 12:07:11.346337 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459fb79b-efda-4902-9e1c-8665d1bfab83 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:07:11.346935 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459fb79b-efda-4902-9e1c-8665d1bfab83 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:07:11.347017 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459fb79b-efda-4902-9e1c-8665d1bfab83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:07:11.385837 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-459fb79b-efda-4902-9e1c-8665d1bfab83 None None] MFA Rules not processed for user `15bc8f7341dd46bb83584f5dc4d5d50d`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:07:11.387639 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-459fb79b-efda-4902-9e1c-8665d1bfab83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:07:11.632907 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1113/2231] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:07:11 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 12:07:11.640522 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8264345e-c4d1-414a-9265-759d4677f29a service nova] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:07:11.656456 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8264345e-c4d1-414a-9265-759d4677f29a service nova] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:07:11.665599 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44a9c5a-4fd4-4b8c-9456-57be93920201 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bad5e1cb306b46818096940acc3810b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:07:11.666829 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44a9c5a-4fd4-4b8c-9456-57be93920201 service nova] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:07:11.667383 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44a9c5a-4fd4-4b8c-9456-57be93920201 service nova] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:07:11.667902 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44a9c5a-4fd4-4b8c-9456-57be93920201 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:07:11.761118 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44a9c5a-4fd4-4b8c-9456-57be93920201 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:07:11.762948 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44a9c5a-4fd4-4b8c-9456-57be93920201 service nova] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:07:11.773580 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1119/2232] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:07:11 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 11 12:07:13.490961 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fabda32-a31c-4b9f-a6eb-0c969833a95e None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:07:13.491552 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fabda32-a31c-4b9f-a6eb-0c969833a95e None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:07:13.491951 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fabda32-a31c-4b9f-a6eb-0c969833a95e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:07:13.527427 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-7fabda32-a31c-4b9f-a6eb-0c969833a95e None None] MFA Rules not processed for user `edb722b67a804915bf38f34b67845bc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:07:13.529534 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-7fabda32-a31c-4b9f-a6eb-0c969833a95e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:07:13.542497 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1114/2233] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:07:13 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 12:07:21.768554 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=76 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:07:23.516977 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=53,keystone:INSERT=2 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:07:30.338880 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44a9c5a-4fd4-4b8c-9456-57be93920201 service nova] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:07:30.340336 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-b44a9c5a-4fd4-4b8c-9456-57be93920201 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:07:30.410818 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44a9c5a-4fd4-4b8c-9456-57be93920201 service nova] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:07:30.414762 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4d19e7-45ac-46e5-ba75-3e982ba78490 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e563fe174db4cc19ae6bf8162cc3d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:07:30.415395 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4d19e7-45ac-46e5-ba75-3e982ba78490 service neutron] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:07:30.415674 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4d19e7-45ac-46e5-ba75-3e982ba78490 service neutron] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:07:30.415950 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4d19e7-45ac-46e5-ba75-3e982ba78490 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:07:30.417810 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-0c4d19e7-45ac-46e5-ba75-3e982ba78490 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:07:30.504122 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4d19e7-45ac-46e5-ba75-3e982ba78490 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:07:30.505684 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4d19e7-45ac-46e5-ba75-3e982ba78490 service neutron] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:07:30.516494 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1120/2234] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:07:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:07:34.649254 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024445eb-9ecb-476a-a6d6-4b259631c55e None None] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:07:34.650296 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024445eb-9ecb-476a-a6d6-4b259631c55e None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:07:34.650296 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024445eb-9ecb-476a-a6d6-4b259631c55e None None] PATH_INFO: `/` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:07:34.653044 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1115/2235] 10.222.0.59 () {54 vars in 921 bytes} [Tue Oct 11 12:07:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 12:07:34.692914 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30344e60-1f4d-4f80-aa0b-a84bdaea7d13 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:07:34.693649 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30344e60-1f4d-4f80-aa0b-a84bdaea7d13 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:07:34.693649 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30344e60-1f4d-4f80-aa0b-a84bdaea7d13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:07:34.717862 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-30344e60-1f4d-4f80-aa0b-a84bdaea7d13 None None] MFA Rules not processed for user `f272cdd1bcc04b4e8261ed533cfd4fd7`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:07:34.719764 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-30344e60-1f4d-4f80-aa0b-a84bdaea7d13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:07:34.763268 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024445eb-9ecb-476a-a6d6-4b259631c55e None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:07:34.765229 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-024445eb-9ecb-476a-a6d6-4b259631c55e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:07:34.783530 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024445eb-9ecb-476a-a6d6-4b259631c55e None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:07:34.794128 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7048d06c-c51d-44b3-a9b6-4ab301562993 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:07:34.795575 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7048d06c-c51d-44b3-a9b6-4ab301562993 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:07:34.796031 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7048d06c-c51d-44b3-a9b6-4ab301562993 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:07:34.796733 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7048d06c-c51d-44b3-a9b6-4ab301562993 admin admin] PATH_INFO: `/v3/users/c58107eee1b94541a330c26d7b82a53b` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:07:34.798916 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7048d06c-c51d-44b3-a9b6-4ab301562993 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c58107eee1b94541a330c26d7b82a53b)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:07:34.802770 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7048d06c-c51d-44b3-a9b6-4ab301562993 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:07:34.817266 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1121/2236] 10.222.0.59 () {58 vars in 1018 bytes} [Tue Oct 11 12:07:34 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 12:07:34.829411 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30344e60-1f4d-4f80-aa0b-a84bdaea7d13 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:07:34.830821 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-30344e60-1f4d-4f80-aa0b-a84bdaea7d13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:07:34.844619 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30344e60-1f4d-4f80-aa0b-a84bdaea7d13 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:07:34.851762 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0f5420-33a0-4429-8e68-4b9bcba0c474 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f272cdd1bcc04b4e8261ed533cfd4fd7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:07:34.852498 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0f5420-33a0-4429-8e68-4b9bcba0c474 service placement] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:07:34.852792 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0f5420-33a0-4429-8e68-4b9bcba0c474 service placement] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:07:34.853103 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0f5420-33a0-4429-8e68-4b9bcba0c474 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:07:34.864249 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0f5420-33a0-4429-8e68-4b9bcba0c474 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:07:34.865983 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0f5420-33a0-4429-8e68-4b9bcba0c474 service placement] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:07:34.876921 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1122/2237] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:07:34 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 11 12:07:34.888347 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-7048d06c-c51d-44b3-a9b6-4ab301562993 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c58107eee1b94541a330c26d7b82a53b'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:07:34.916499 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-7048d06c-c51d-44b3-a9b6-4ab301562993 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c58107eee1b94541a330c26d7b82a53b'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:07:34.929498 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-7048d06c-c51d-44b3-a9b6-4ab301562993 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c58107eee1b94541a330c26d7b82a53b'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:07:34.930533 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-7048d06c-c51d-44b3-a9b6-4ab301562993 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c58107eee1b94541a330c26d7b82a53b'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:07:34.946003 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-7048d06c-c51d-44b3-a9b6-4ab301562993 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c58107eee1b94541a330c26d7b82a53b'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:07:34.957672 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1116/2238] 10.222.0.59 () {58 vars in 1224 bytes} [Tue Oct 11 12:07:34 2022] DELETE /identity/v3/users/c58107eee1b94541a330c26d7b82a53b => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 12:07:35.169967 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0f5420-33a0-4429-8e68-4b9bcba0c474 service placement] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:07:35.185496 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0f5420-33a0-4429-8e68-4b9bcba0c474 service placement] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:07:35.234586 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:07:35.273698 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6e33d1-c824-4ab8-95f4-e7489e4cddba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:07:35.274507 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6e33d1-c824-4ab8-95f4-e7489e4cddba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:07:35.275014 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6e33d1-c824-4ab8-95f4-e7489e4cddba admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:07:35.275484 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6e33d1-c824-4ab8-95f4-e7489e4cddba admin admin] PATH_INFO: `/v3/projects/e3e6a648f46646cea42b910d02e0628f` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:07:35.276934 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6e33d1-c824-4ab8-95f4-e7489e4cddba admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3e6a648f46646cea42b910d02e0628f)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:07:35.279225 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6e33d1-c824-4ab8-95f4-e7489e4cddba admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:07:35.348288 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-ec6e33d1-c824-4ab8-95f4-e7489e4cddba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3e6a648f46646cea42b910d02e0628f', 'request_id': 'req-ec6e33d1-c824-4ab8-95f4-e7489e4cddba'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:07:35.356413 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-ec6e33d1-c824-4ab8-95f4-e7489e4cddba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3e6a648f46646cea42b910d02e0628f', 'request_id': 'req-ec6e33d1-c824-4ab8-95f4-e7489e4cddba'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:07:35.364183 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-ec6e33d1-c824-4ab8-95f4-e7489e4cddba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3e6a648f46646cea42b910d02e0628f', 'request_id': 'req-ec6e33d1-c824-4ab8-95f4-e7489e4cddba'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:07:35.381335 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-ec6e33d1-c824-4ab8-95f4-e7489e4cddba admin admin] The token cache is being invalidate because project e3e6a648f46646cea42b910d02e0628f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81432) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 12:07:35.381650 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-ec6e33d1-c824-4ab8-95f4-e7489e4cddba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:07:35.382541 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-ec6e33d1-c824-4ab8-95f4-e7489e4cddba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:07:35.384334 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1123/2239] 10.222.0.59 () {58 vars in 1233 bytes} [Tue Oct 11 12:07:35 2022] DELETE /identity/v3/projects/e3e6a648f46646cea42b910d02e0628f => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:07:35.390111 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f3e074-0582-4d1d-93ac-023a6616c033 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:07:35.390485 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f3e074-0582-4d1d-93ac-023a6616c033 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:07:35.390867 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f3e074-0582-4d1d-93ac-023a6616c033 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:07:35.416256 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-a8f3e074-0582-4d1d-93ac-023a6616c033 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:07:35.417512 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-a8f3e074-0582-4d1d-93ac-023a6616c033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:07:35.497245 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1117/2240] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:07:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:07:35.511173 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0188cf4-91ee-49b7-a978-9cfccc02d9c0 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:07:35.511697 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0188cf4-91ee-49b7-a978-9cfccc02d9c0 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:07:35.512212 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0188cf4-91ee-49b7-a978-9cfccc02d9c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:07:35.548698 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-e0188cf4-91ee-49b7-a978-9cfccc02d9c0 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:07:35.550891 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-e0188cf4-91ee-49b7-a978-9cfccc02d9c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:07:35.565374 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1124/2241] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:07:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:07:35.570806 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f3e074-0582-4d1d-93ac-023a6616c033 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:07:35.582186 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f3e074-0582-4d1d-93ac-023a6616c033 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:07:35.587791 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61122cab-7b28-4006-bf01-230fe6fa95a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:07:35.588671 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61122cab-7b28-4006-bf01-230fe6fa95a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:07:35.589051 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61122cab-7b28-4006-bf01-230fe6fa95a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:07:35.589391 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61122cab-7b28-4006-bf01-230fe6fa95a4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:07:35.590172 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61122cab-7b28-4006-bf01-230fe6fa95a4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 12:07:35.590533 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61122cab-7b28-4006-bf01-230fe6fa95a4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:07:35.592358 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61122cab-7b28-4006-bf01-230fe6fa95a4 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:07:35.607075 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1118/2242] 10.222.0.59 () {58 vars in 1153 bytes} [Tue Oct 11 12:07:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 12:07:35.611968 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0188cf4-91ee-49b7-a978-9cfccc02d9c0 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:07:35.623680 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0188cf4-91ee-49b7-a978-9cfccc02d9c0 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:07:35.629311 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7b46a2-790f-4804-a144-869b246ee5c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:07:35.630009 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7b46a2-790f-4804-a144-869b246ee5c4 admin admin] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:07:35.630377 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7b46a2-790f-4804-a144-869b246ee5c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:07:35.630740 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7b46a2-790f-4804-a144-869b246ee5c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:07:35.631564 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7b46a2-790f-4804-a144-869b246ee5c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:07:35.633621 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7b46a2-790f-4804-a144-869b246ee5c4 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:07:35.669195 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1125/2243] 10.222.0.59 () {60 vars in 1153 bytes} [Tue Oct 11 12:07:35 2022] POST /identity/v3/projects => generated 387 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:07:35.674854 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61122cab-7b28-4006-bf01-230fe6fa95a4 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:07:35.690939 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61122cab-7b28-4006-bf01-230fe6fa95a4 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:07:35.793473 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7c2ab0-a334-4846-a2d8-98478c0d7d00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:07:35.794854 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7c2ab0-a334-4846-a2d8-98478c0d7d00 admin admin] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:07:35.795325 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7c2ab0-a334-4846-a2d8-98478c0d7d00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:07:35.795836 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7c2ab0-a334-4846-a2d8-98478c0d7d00 admin admin] PATH_INFO: `/v3/users` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:07:35.797266 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7c2ab0-a334-4846-a2d8-98478c0d7d00 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:07:35.801649 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7c2ab0-a334-4846-a2d8-98478c0d7d00 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:07:35.819515 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:07:35.840794 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1119/2244] 10.222.0.59 () {60 vars in 1144 bytes} [Tue Oct 11 12:07:35 2022] POST /identity/v3/users => generated 294 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:07:35.847207 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7b46a2-790f-4804-a144-869b246ee5c4 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:07:35.860449 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7b46a2-790f-4804-a144-869b246ee5c4 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:07:35.866464 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8723f16-bc24-436e-9e54-cb0998a3a8ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:07:35.867353 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8723f16-bc24-436e-9e54-cb0998a3a8ac admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:07:35.867686 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8723f16-bc24-436e-9e54-cb0998a3a8ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:07:35.868049 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8723f16-bc24-436e-9e54-cb0998a3a8ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:07:35.868910 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8723f16-bc24-436e-9e54-cb0998a3a8ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:07:35.871069 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8723f16-bc24-436e-9e54-cb0998a3a8ac admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:07:35.884237 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1126/2245] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:07:35 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:07:35.889727 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7c2ab0-a334-4846-a2d8-98478c0d7d00 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:07:35.903855 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7c2ab0-a334-4846-a2d8-98478c0d7d00 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:07:35.911699 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bac207e-11db-4db7-8e99-be13aa6936e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:07:35.912500 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bac207e-11db-4db7-8e99-be13aa6936e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:07:35.912826 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bac207e-11db-4db7-8e99-be13aa6936e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:07:35.913152 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bac207e-11db-4db7-8e99-be13aa6936e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:07:35.913949 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bac207e-11db-4db7-8e99-be13aa6936e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:07:35.916026 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bac207e-11db-4db7-8e99-be13aa6936e4 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:07:35.932639 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1120/2246] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:07:35 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:07:35.938079 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8723f16-bc24-436e-9e54-cb0998a3a8ac admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:07:35.952244 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8723f16-bc24-436e-9e54-cb0998a3a8ac admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:07:35.959148 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7498bf95-cf5a-4a49-a6bd-ccaef4b067d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:07:35.959986 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7498bf95-cf5a-4a49-a6bd-ccaef4b067d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:07:35.960444 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7498bf95-cf5a-4a49-a6bd-ccaef4b067d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:07:35.961216 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7498bf95-cf5a-4a49-a6bd-ccaef4b067d3 admin admin] PATH_INFO: `/v3/projects/fee1ecae514e4c79a4f0fe601391971d/users/abfa19ab0a624de6ab082e8b1e57a220/roles/de2d2bd26fff4748a79990f97a107164` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:07:35.985417 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7498bf95-cf5a-4a49-a6bd-ccaef4b067d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fee1ecae514e4c79a4f0fe601391971d, user_id=abfa19ab0a624de6ab082e8b1e57a220, role_id=de2d2bd26fff4748a79990f97a107164)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:07:35.988925 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7498bf95-cf5a-4a49-a6bd-ccaef4b067d3 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:07:36.007927 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1127/2247] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 12:07:35 2022] PUT /identity/v3/projects/fee1ecae514e4c79a4f0fe601391971d/users/abfa19ab0a624de6ab082e8b1e57a220/roles/de2d2bd26fff4748a79990f97a107164 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:07:36.014592 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03b5f65-bcd0-4dff-80a6-6463090281a4 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:07:36.014897 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03b5f65-bcd0-4dff-80a6-6463090281a4 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:07:36.015203 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03b5f65-bcd0-4dff-80a6-6463090281a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:07:36.039468 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-c03b5f65-bcd0-4dff-80a6-6463090281a4 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:07:36.040845 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-c03b5f65-bcd0-4dff-80a6-6463090281a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:07:36.119306 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1121/2248] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:07:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:07:36.131188 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7498bf95-cf5a-4a49-a6bd-ccaef4b067d3 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:07:36.133236 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-7498bf95-cf5a-4a49-a6bd-ccaef4b067d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:07:36.218773 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7498bf95-cf5a-4a49-a6bd-ccaef4b067d3 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:07:36.224918 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa586452-1a9c-4bca-934f-e6cec4792d0a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e563fe174db4cc19ae6bf8162cc3d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:07:36.225566 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa586452-1a9c-4bca-934f-e6cec4792d0a service neutron] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:07:36.225843 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa586452-1a9c-4bca-934f-e6cec4792d0a service neutron] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:07:36.226116 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa586452-1a9c-4bca-934f-e6cec4792d0a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:07:36.237709 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa586452-1a9c-4bca-934f-e6cec4792d0a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:07:36.239415 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa586452-1a9c-4bca-934f-e6cec4792d0a service neutron] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:07:36.251520 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1128/2249] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:07:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 11 12:07:42.728009 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a7a090-e061-4065-904f-0e91888ecf6d None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:07:42.728671 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a7a090-e061-4065-904f-0e91888ecf6d None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:07:42.729268 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a7a090-e061-4065-904f-0e91888ecf6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:07:42.799336 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-b5a7a090-e061-4065-904f-0e91888ecf6d None None] MFA Rules not processed for user `abfa19ab0a624de6ab082e8b1e57a220`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:07:42.801790 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-b5a7a090-e061-4065-904f-0e91888ecf6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:07:42.942281 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1122/2250] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:07:42 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:07:42.972538 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa586452-1a9c-4bca-934f-e6cec4792d0a service neutron] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:07:42.993885 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-fa586452-1a9c-4bca-934f-e6cec4792d0a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:07:42.994506 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:07:43.152385 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa586452-1a9c-4bca-934f-e6cec4792d0a service neutron] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:07:43.163547 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15630435-ab62-433f-a99f-3d4ddf1a7e94 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bad5e1cb306b46818096940acc3810b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:07:43.164765 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15630435-ab62-433f-a99f-3d4ddf1a7e94 service nova] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:07:43.165345 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15630435-ab62-433f-a99f-3d4ddf1a7e94 service nova] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:07:43.165911 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15630435-ab62-433f-a99f-3d4ddf1a7e94 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:07:43.182742 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15630435-ab62-433f-a99f-3d4ddf1a7e94 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:07:43.186221 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15630435-ab62-433f-a99f-3d4ddf1a7e94 service nova] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:07:43.209662 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1129/2251] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:07:42 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 11 12:07:44.934150 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e587df-0b77-4170-a03f-e57ed36c3564 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:07:44.934761 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e587df-0b77-4170-a03f-e57ed36c3564 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:07:44.934967 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e587df-0b77-4170-a03f-e57ed36c3564 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:07:44.955942 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-69e587df-0b77-4170-a03f-e57ed36c3564 None None] MFA Rules not processed for user `abfa19ab0a624de6ab082e8b1e57a220`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:07:44.957817 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-69e587df-0b77-4170-a03f-e57ed36c3564 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:07:44.969722 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1123/2252] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:07:44 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:07:53.196662 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=33 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:07:54.949462 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:INSERT=3,keystone:SELECT=69 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:07:57.362016 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15630435-ab62-433f-a99f-3d4ddf1a7e94 service nova] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:07:57.386025 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15630435-ab62-433f-a99f-3d4ddf1a7e94 service nova] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:07:57.389923 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662982d3-d592-4b50-84df-0ca6fad3bebd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e563fe174db4cc19ae6bf8162cc3d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:07:57.390545 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662982d3-d592-4b50-84df-0ca6fad3bebd service neutron] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:07:57.390861 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662982d3-d592-4b50-84df-0ca6fad3bebd service neutron] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:07:57.391137 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662982d3-d592-4b50-84df-0ca6fad3bebd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:07:57.393324 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-662982d3-d592-4b50-84df-0ca6fad3bebd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:07:57.498793 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662982d3-d592-4b50-84df-0ca6fad3bebd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:07:57.501026 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662982d3-d592-4b50-84df-0ca6fad3bebd service neutron] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:07:57.512108 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1130/2253] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:07:57 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:07:59.842359 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e587df-0b77-4170-a03f-e57ed36c3564 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:07:59.933154 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e587df-0b77-4170-a03f-e57ed36c3564 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:07:59.941877 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfdec55-a73a-4767-aef4-693217edb90b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bad5e1cb306b46818096940acc3810b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:07:59.942999 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfdec55-a73a-4767-aef4-693217edb90b service nova] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:07:59.944434 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfdec55-a73a-4767-aef4-693217edb90b service nova] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:07:59.944993 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfdec55-a73a-4767-aef4-693217edb90b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:07:59.947484 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-5bfdec55-a73a-4767-aef4-693217edb90b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:07:59.985035 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfdec55-a73a-4767-aef4-693217edb90b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:07:59.987627 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfdec55-a73a-4767-aef4-693217edb90b service nova] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:08:00.022039 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1124/2254] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:07:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:08:07.507786 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=48 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:08:09.999848 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=29 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:08:13.906245 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662982d3-d592-4b50-84df-0ca6fad3bebd service neutron] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:08:13.908014 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-662982d3-d592-4b50-84df-0ca6fad3bebd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:08:14.012205 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662982d3-d592-4b50-84df-0ca6fad3bebd service neutron] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:08:14.017329 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed7e8af-a712-482d-983c-64b5c6bec3d6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f272cdd1bcc04b4e8261ed533cfd4fd7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:08:14.018293 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed7e8af-a712-482d-983c-64b5c6bec3d6 service placement] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:08:14.018577 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed7e8af-a712-482d-983c-64b5c6bec3d6 service placement] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:08:14.018866 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed7e8af-a712-482d-983c-64b5c6bec3d6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:08:14.020614 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-8ed7e8af-a712-482d-983c-64b5c6bec3d6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:08:14.032323 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed7e8af-a712-482d-983c-64b5c6bec3d6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:08:14.033931 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed7e8af-a712-482d-983c-64b5c6bec3d6 service placement] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:08:14.044436 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1131/2255] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:08:13 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:08:17.592016 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc393825-72e8-4a47-961f-35d18bcb96c6 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:08:17.592518 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc393825-72e8-4a47-961f-35d18bcb96c6 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:08:17.592631 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc393825-72e8-4a47-961f-35d18bcb96c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:08:17.612291 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-cc393825-72e8-4a47-961f-35d18bcb96c6 None None] MFA Rules not processed for user `41d3680dda7c4cc8aeb6d086efca0775`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:08:17.613504 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-cc393825-72e8-4a47-961f-35d18bcb96c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:08:17.674263 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1125/2256] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:08:17 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:08:24.040576 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=41 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:08:27.667608 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=25 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:08:30.068153 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccda86b-4c13-4ea3-a94a-3365c4f56ec3 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:08:30.068760 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccda86b-4c13-4ea3-a94a-3365c4f56ec3 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:08:30.068760 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccda86b-4c13-4ea3-a94a-3365c4f56ec3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:08:30.089235 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-eccda86b-4c13-4ea3-a94a-3365c4f56ec3 None None] MFA Rules not processed for user `a78e40d4eb554ca49eeec530a7adf428`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:08:30.090467 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-eccda86b-4c13-4ea3-a94a-3365c4f56ec3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:08:30.150489 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1132/2257] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:08:30 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:08:40.143637 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=25 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:09:49.615391 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e7f41f-066f-4e8e-97ef-ebdf9db0e0e7 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:09:49.615935 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e7f41f-066f-4e8e-97ef-ebdf9db0e0e7 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:09:49.615935 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e7f41f-066f-4e8e-97ef-ebdf9db0e0e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:09:49.647161 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-20e7f41f-066f-4e8e-97ef-ebdf9db0e0e7 None None] MFA Rules not processed for user `8a19eceb62ce4157a4c5fe1866a89df9`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:09:49.648392 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-20e7f41f-066f-4e8e-97ef-ebdf9db0e0e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:09:49.718856 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1126/2258] 10.222.0.59 () {58 vars in 1009 bytes} [Tue Oct 11 12:09:49 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:09:49.771235 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccda86b-4c13-4ea3-a94a-3365c4f56ec3 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:09:49.773234 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-eccda86b-4c13-4ea3-a94a-3365c4f56ec3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:09:49.846784 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccda86b-4c13-4ea3-a94a-3365c4f56ec3 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:09:49.852364 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ea3dd5-21b3-4b60-969c-9887edfda46a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76e7fee1993a40d5ba4295e7daeb3e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:09:49.853007 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ea3dd5-21b3-4b60-969c-9887edfda46a service swift] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:09:49.853284 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ea3dd5-21b3-4b60-969c-9887edfda46a service swift] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:09:49.853559 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ea3dd5-21b3-4b60-969c-9887edfda46a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:09:49.863856 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ea3dd5-21b3-4b60-969c-9887edfda46a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:09:49.866299 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ea3dd5-21b3-4b60-969c-9887edfda46a service swift] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:09:49.879587 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1133/2259] 10.222.0.59 () {60 vars in 1384 bytes} [Tue Oct 11 12:09:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 11 12:09:56.522866 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e7f41f-066f-4e8e-97ef-ebdf9db0e0e7 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:09:56.543408 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e7f41f-066f-4e8e-97ef-ebdf9db0e0e7 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:09:56.548808 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af86c627-7ff6-4834-b708-d7dd09d17fbf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e563fe174db4cc19ae6bf8162cc3d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:09:56.548808 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af86c627-7ff6-4834-b708-d7dd09d17fbf service neutron] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:09:56.548808 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af86c627-7ff6-4834-b708-d7dd09d17fbf service neutron] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:09:56.548808 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af86c627-7ff6-4834-b708-d7dd09d17fbf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:09:56.550657 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-af86c627-7ff6-4834-b708-d7dd09d17fbf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:09:56.568325 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af86c627-7ff6-4834-b708-d7dd09d17fbf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:09:56.569971 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af86c627-7ff6-4834-b708-d7dd09d17fbf service neutron] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:09:56.581509 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1127/2260] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:09:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:09:57.797792 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ea3dd5-21b3-4b60-969c-9887edfda46a service swift] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:09:57.799673 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-41ea3dd5-21b3-4b60-969c-9887edfda46a service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:09:57.825950 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ea3dd5-21b3-4b60-969c-9887edfda46a service swift] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:09:57.829902 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec5c7e8-f449-4622-b9ec-7fd4b64f754f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e563fe174db4cc19ae6bf8162cc3d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:09:57.830611 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec5c7e8-f449-4622-b9ec-7fd4b64f754f service neutron] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:09:57.830912 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec5c7e8-f449-4622-b9ec-7fd4b64f754f service neutron] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:09:57.831179 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec5c7e8-f449-4622-b9ec-7fd4b64f754f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:09:57.833562 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-bec5c7e8-f449-4622-b9ec-7fd4b64f754f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:09:57.850788 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec5c7e8-f449-4622-b9ec-7fd4b64f754f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:09:57.852246 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec5c7e8-f449-4622-b9ec-7fd4b64f754f service neutron] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:09:57.868593 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1134/2261] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:09:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:10:05.967125 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197577b6-82d4-4391-8ac4-178adf83d495 None None] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:10:05.967644 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197577b6-82d4-4391-8ac4-178adf83d495 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:10:05.967739 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197577b6-82d4-4391-8ac4-178adf83d495 None None] PATH_INFO: `/` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:10:05.969294 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1128/2262] 10.222.0.59 () {54 vars in 921 bytes} [Tue Oct 11 12:10:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 12:10:05.978496 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695fd0ce-0b00-4e56-92fc-5624599d1c0f None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:10:05.979036 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695fd0ce-0b00-4e56-92fc-5624599d1c0f None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:10:05.979245 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695fd0ce-0b00-4e56-92fc-5624599d1c0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:10:06.034109 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-695fd0ce-0b00-4e56-92fc-5624599d1c0f None None] MFA Rules not processed for user `bad5e1cb306b46818096940acc3810b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:10:06.035592 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-695fd0ce-0b00-4e56-92fc-5624599d1c0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:10:06.045523 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1135/2263] 10.222.0.59 () {58 vars in 1018 bytes} [Tue Oct 11 12:10:05 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 12:10:06.057574 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197577b6-82d4-4391-8ac4-178adf83d495 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:10:06.059333 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-197577b6-82d4-4391-8ac4-178adf83d495 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:10:06.125183 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197577b6-82d4-4391-8ac4-178adf83d495 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:10:06.129132 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b352127-a3f7-4490-b073-c1a3887d6354 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91a00658b79a42bd91a22a2508b5de34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:10:06.129743 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b352127-a3f7-4490-b073-c1a3887d6354 service glance] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:10:06.130066 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b352127-a3f7-4490-b073-c1a3887d6354 service glance] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:10:06.130338 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b352127-a3f7-4490-b073-c1a3887d6354 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:10:06.141526 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b352127-a3f7-4490-b073-c1a3887d6354 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:10:06.143033 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b352127-a3f7-4490-b073-c1a3887d6354 service glance] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:10:06.153813 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1129/2264] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:10:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:10:06.231629 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5164b639-942a-41c6-b160-4738d2008cad None None] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:10:06.231930 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5164b639-942a-41c6-b160-4738d2008cad None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:10:06.232293 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5164b639-942a-41c6-b160-4738d2008cad None None] PATH_INFO: `/` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:10:06.234319 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1136/2265] 10.222.0.59 () {54 vars in 921 bytes} [Tue Oct 11 12:10:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 12:10:06.239931 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9ebc02-dc1a-41a2-b875-4dec2486132d None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:10:06.240337 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9ebc02-dc1a-41a2-b875-4dec2486132d None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:10:06.240740 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9ebc02-dc1a-41a2-b875-4dec2486132d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:10:06.266894 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-4b9ebc02-dc1a-41a2-b875-4dec2486132d None None] MFA Rules not processed for user `2e563fe174db4cc19ae6bf8162cc3d23`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:10:06.268223 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-4b9ebc02-dc1a-41a2-b875-4dec2486132d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:10:06.275684 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1130/2266] 10.222.0.59 () {58 vars in 1018 bytes} [Tue Oct 11 12:10:06 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:10:06.287710 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5164b639-942a-41c6-b160-4738d2008cad None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:10:06.300450 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5164b639-942a-41c6-b160-4738d2008cad None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:10:06.304301 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a19f21a-636f-4126-8472-3d5057451c7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e563fe174db4cc19ae6bf8162cc3d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:10:06.304912 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a19f21a-636f-4126-8472-3d5057451c7b service neutron] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:10:06.305173 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a19f21a-636f-4126-8472-3d5057451c7b service neutron] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:10:06.305440 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a19f21a-636f-4126-8472-3d5057451c7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:10:06.313010 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a19f21a-636f-4126-8472-3d5057451c7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:10:06.314509 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a19f21a-636f-4126-8472-3d5057451c7b service neutron] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:10:06.322182 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1137/2267] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:10:06 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:10:06.417272 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92975a94-d372-46a1-bfd0-162e1af58e57 None None] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:10:06.417539 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92975a94-d372-46a1-bfd0-162e1af58e57 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:10:06.417839 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92975a94-d372-46a1-bfd0-162e1af58e57 None None] PATH_INFO: `/` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:10:06.419427 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1131/2268] 10.222.0.59 () {54 vars in 921 bytes} [Tue Oct 11 12:10:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 12:10:06.425030 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e270f5c6-0021-4d37-b03e-80e3698c6535 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:10:06.425316 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e270f5c6-0021-4d37-b03e-80e3698c6535 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:10:06.425584 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e270f5c6-0021-4d37-b03e-80e3698c6535 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:10:06.447067 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-e270f5c6-0021-4d37-b03e-80e3698c6535 None None] MFA Rules not processed for user `bad5e1cb306b46818096940acc3810b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:10:06.448325 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-e270f5c6-0021-4d37-b03e-80e3698c6535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:10:06.459889 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1138/2269] 10.222.0.59 () {58 vars in 1018 bytes} [Tue Oct 11 12:10:06 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:10:06.470919 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92975a94-d372-46a1-bfd0-162e1af58e57 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:10:06.483187 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92975a94-d372-46a1-bfd0-162e1af58e57 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:10:06.487330 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152dbaa7-749a-4e8a-a39f-47d2536a239b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e563fe174db4cc19ae6bf8162cc3d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:10:06.487928 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152dbaa7-749a-4e8a-a39f-47d2536a239b service neutron] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:10:06.488173 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152dbaa7-749a-4e8a-a39f-47d2536a239b service neutron] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:10:06.488468 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152dbaa7-749a-4e8a-a39f-47d2536a239b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:10:06.500254 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152dbaa7-749a-4e8a-a39f-47d2536a239b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:10:06.501548 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152dbaa7-749a-4e8a-a39f-47d2536a239b service neutron] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:10:06.513233 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1132/2270] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:10:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:10:12.835734 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04f9c1d-d01b-4aec-8348-eeb9440d8d17 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:10:12.836292 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04f9c1d-d01b-4aec-8348-eeb9440d8d17 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:10:12.836590 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04f9c1d-d01b-4aec-8348-eeb9440d8d17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:10:12.844593 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=66 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:10:12.860984 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-c04f9c1d-d01b-4aec-8348-eeb9440d8d17 None None] MFA Rules not processed for user `8a19eceb62ce4157a4c5fe1866a89df9`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:10:12.862295 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-c04f9c1d-d01b-4aec-8348-eeb9440d8d17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:10:12.869728 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1139/2271] 10.222.0.59 () {58 vars in 1009 bytes} [Tue Oct 11 12:10:12 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:10:12.883350 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152dbaa7-749a-4e8a-a39f-47d2536a239b service neutron] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:10:12.896082 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152dbaa7-749a-4e8a-a39f-47d2536a239b service neutron] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:10:12.901167 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d580ddf-bc28-464f-b774-28d85e246247 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76e7fee1993a40d5ba4295e7daeb3e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:10:12.901877 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d580ddf-bc28-464f-b774-28d85e246247 service swift] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:10:12.902222 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d580ddf-bc28-464f-b774-28d85e246247 service swift] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:10:12.902580 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d580ddf-bc28-464f-b774-28d85e246247 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:10:12.911111 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d580ddf-bc28-464f-b774-28d85e246247 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:10:12.912801 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d580ddf-bc28-464f-b774-28d85e246247 service swift] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:10:12.921169 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1133/2272] 10.222.0.59 () {60 vars in 1384 bytes} [Tue Oct 11 12:10:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 11 12:10:22.856028 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=2 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:10:22.917764 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=80 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:10:34.364511 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04f9c1d-d01b-4aec-8348-eeb9440d8d17 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:10:34.374581 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04f9c1d-d01b-4aec-8348-eeb9440d8d17 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:10:34.379723 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d75534-9a7f-4f91-a2da-c314ab74951b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e563fe174db4cc19ae6bf8162cc3d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:10:34.380147 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d75534-9a7f-4f91-a2da-c314ab74951b service neutron] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:10:34.380454 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d75534-9a7f-4f91-a2da-c314ab74951b service neutron] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:10:34.380776 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d75534-9a7f-4f91-a2da-c314ab74951b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:10:34.382496 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-50d75534-9a7f-4f91-a2da-c314ab74951b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:10:34.393096 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d75534-9a7f-4f91-a2da-c314ab74951b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:10:34.394549 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d75534-9a7f-4f91-a2da-c314ab74951b service neutron] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:10:34.404083 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1140/2273] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:10:34 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:10:35.111404 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d580ddf-bc28-464f-b774-28d85e246247 service swift] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:10:35.121850 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d580ddf-bc28-464f-b774-28d85e246247 service swift] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:10:35.126406 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca4ab8d-a314-4d2c-a8da-b1bc3f14480c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e563fe174db4cc19ae6bf8162cc3d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:10:35.127174 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca4ab8d-a314-4d2c-a8da-b1bc3f14480c service neutron] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:10:35.127457 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca4ab8d-a314-4d2c-a8da-b1bc3f14480c service neutron] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:10:35.127741 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca4ab8d-a314-4d2c-a8da-b1bc3f14480c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:10:35.129927 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-dca4ab8d-a314-4d2c-a8da-b1bc3f14480c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:10:35.169945 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca4ab8d-a314-4d2c-a8da-b1bc3f14480c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:10:35.171762 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca4ab8d-a314-4d2c-a8da-b1bc3f14480c service neutron] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:10:35.192786 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1134/2274] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:10:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:10:36.045501 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a4b072-afe9-49aa-81ef-94b7a7e00273 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:10:36.046462 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a4b072-afe9-49aa-81ef-94b7a7e00273 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:10:36.046900 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a4b072-afe9-49aa-81ef-94b7a7e00273 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:10:36.099208 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-c2a4b072-afe9-49aa-81ef-94b7a7e00273 None None] MFA Rules not processed for user `8a19eceb62ce4157a4c5fe1866a89df9`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:10:36.100465 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-c2a4b072-afe9-49aa-81ef-94b7a7e00273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:10:36.118046 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1141/2275] 10.222.0.59 () {58 vars in 967 bytes} [Tue Oct 11 12:10:36 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:10:36.132615 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca4ab8d-a314-4d2c-a8da-b1bc3f14480c service neutron] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:10:36.169316 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca4ab8d-a314-4d2c-a8da-b1bc3f14480c service neutron] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:10:36.175060 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466d0c20-2ecc-43fa-80fe-616dac3b8966 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76e7fee1993a40d5ba4295e7daeb3e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:10:36.175690 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466d0c20-2ecc-43fa-80fe-616dac3b8966 service swift] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:10:36.175965 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466d0c20-2ecc-43fa-80fe-616dac3b8966 service swift] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:10:36.176239 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466d0c20-2ecc-43fa-80fe-616dac3b8966 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:10:36.188049 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466d0c20-2ecc-43fa-80fe-616dac3b8966 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:10:36.190217 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466d0c20-2ecc-43fa-80fe-616dac3b8966 service swift] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:10:36.200794 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1135/2276] 10.222.0.59 () {60 vars in 1384 bytes} [Tue Oct 11 12:10:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 11 12:10:39.216292 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a4b072-afe9-49aa-81ef-94b7a7e00273 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:10:39.217944 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-c2a4b072-afe9-49aa-81ef-94b7a7e00273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:10:39.233150 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a4b072-afe9-49aa-81ef-94b7a7e00273 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:10:39.238030 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875e86ad-84e8-4017-aca4-b31ce9a7eaf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:10:39.238719 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875e86ad-84e8-4017-aca4-b31ce9a7eaf0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:10:39.238984 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875e86ad-84e8-4017-aca4-b31ce9a7eaf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:10:39.239481 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875e86ad-84e8-4017-aca4-b31ce9a7eaf0 admin admin] PATH_INFO: `/v3/users/41d3680dda7c4cc8aeb6d086efca0775` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:10:39.254538 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875e86ad-84e8-4017-aca4-b31ce9a7eaf0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=41d3680dda7c4cc8aeb6d086efca0775)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:10:39.256516 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875e86ad-84e8-4017-aca4-b31ce9a7eaf0 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:10:39.307992 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-875e86ad-84e8-4017-aca4-b31ce9a7eaf0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41d3680dda7c4cc8aeb6d086efca0775'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:10:39.321352 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-875e86ad-84e8-4017-aca4-b31ce9a7eaf0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41d3680dda7c4cc8aeb6d086efca0775'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:10:39.322072 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-875e86ad-84e8-4017-aca4-b31ce9a7eaf0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41d3680dda7c4cc8aeb6d086efca0775'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:10:39.330069 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-875e86ad-84e8-4017-aca4-b31ce9a7eaf0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41d3680dda7c4cc8aeb6d086efca0775'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:10:39.330702 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-875e86ad-84e8-4017-aca4-b31ce9a7eaf0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41d3680dda7c4cc8aeb6d086efca0775'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:10:39.343334 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1142/2277] 10.222.0.59 () {58 vars in 1224 bytes} [Tue Oct 11 12:10:39 2022] DELETE /identity/v3/users/41d3680dda7c4cc8aeb6d086efca0775 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:10:39.528620 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466d0c20-2ecc-43fa-80fe-616dac3b8966 service swift] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:10:39.540214 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466d0c20-2ecc-43fa-80fe-616dac3b8966 service swift] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:10:39.609603 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15027310-5cd1-473a-bc36-316ceedbfe50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:10:39.610150 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15027310-5cd1-473a-bc36-316ceedbfe50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:10:39.610426 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15027310-5cd1-473a-bc36-316ceedbfe50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:10:39.610728 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15027310-5cd1-473a-bc36-316ceedbfe50 admin admin] PATH_INFO: `/v3/projects/c83f971a0ae04288a5c6dc87d3370979` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:10:39.621128 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15027310-5cd1-473a-bc36-316ceedbfe50 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c83f971a0ae04288a5c6dc87d3370979)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:10:39.623096 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15027310-5cd1-473a-bc36-316ceedbfe50 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:10:39.672159 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-15027310-5cd1-473a-bc36-316ceedbfe50 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c83f971a0ae04288a5c6dc87d3370979', 'request_id': 'req-15027310-5cd1-473a-bc36-316ceedbfe50'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:10:39.677596 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-15027310-5cd1-473a-bc36-316ceedbfe50 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c83f971a0ae04288a5c6dc87d3370979', 'request_id': 'req-15027310-5cd1-473a-bc36-316ceedbfe50'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:10:39.688008 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-15027310-5cd1-473a-bc36-316ceedbfe50 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c83f971a0ae04288a5c6dc87d3370979', 'request_id': 'req-15027310-5cd1-473a-bc36-316ceedbfe50'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:10:39.700071 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-15027310-5cd1-473a-bc36-316ceedbfe50 admin admin] The token cache is being invalidate because project c83f971a0ae04288a5c6dc87d3370979 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 12:10:39.700398 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-15027310-5cd1-473a-bc36-316ceedbfe50 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:10:39.701043 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-15027310-5cd1-473a-bc36-316ceedbfe50 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:10:39.702626 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1136/2278] 10.222.0.59 () {58 vars in 1233 bytes} [Tue Oct 11 12:10:39 2022] DELETE /identity/v3/projects/c83f971a0ae04288a5c6dc87d3370979 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:10:39.707901 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e6d3df-ad9d-4228-b639-3d3589732bca None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:10:39.708258 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e6d3df-ad9d-4228-b639-3d3589732bca None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:10:39.708534 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e6d3df-ad9d-4228-b639-3d3589732bca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:10:39.757260 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-48e6d3df-ad9d-4228-b639-3d3589732bca None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:10:39.758478 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-48e6d3df-ad9d-4228-b639-3d3589732bca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:10:39.802169 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:10:39.821012 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1143/2279] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:10:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:10:39.829240 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81db3825-4a4c-46c5-a96b-914907c6d209 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:10:39.829528 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81db3825-4a4c-46c5-a96b-914907c6d209 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:10:39.829835 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81db3825-4a4c-46c5-a96b-914907c6d209 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:10:39.850575 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-81db3825-4a4c-46c5-a96b-914907c6d209 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:10:39.851802 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-81db3825-4a4c-46c5-a96b-914907c6d209 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:10:39.861546 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1137/2280] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:10:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:10:39.866025 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e6d3df-ad9d-4228-b639-3d3589732bca None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:10:39.878394 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e6d3df-ad9d-4228-b639-3d3589732bca None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:10:39.883769 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312ff1fb-89b1-4668-91f9-876782dbd95f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:10:39.884381 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312ff1fb-89b1-4668-91f9-876782dbd95f admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:10:39.884657 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312ff1fb-89b1-4668-91f9-876782dbd95f admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:10:39.884930 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312ff1fb-89b1-4668-91f9-876782dbd95f admin admin] PATH_INFO: `/v3/domains` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:10:39.885584 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312ff1fb-89b1-4668-91f9-876782dbd95f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 12:10:39.885890 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312ff1fb-89b1-4668-91f9-876782dbd95f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:10:39.887577 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312ff1fb-89b1-4668-91f9-876782dbd95f admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:10:39.899304 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1144/2281] 10.222.0.59 () {58 vars in 1153 bytes} [Tue Oct 11 12:10:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 12:10:39.903775 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81db3825-4a4c-46c5-a96b-914907c6d209 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:10:39.913276 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81db3825-4a4c-46c5-a96b-914907c6d209 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:10:39.918249 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0b519a-2bde-4367-8e30-7c4cf0fea0c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:10:39.918884 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0b519a-2bde-4367-8e30-7c4cf0fea0c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:10:39.919164 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0b519a-2bde-4367-8e30-7c4cf0fea0c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:10:39.919434 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0b519a-2bde-4367-8e30-7c4cf0fea0c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:10:39.920142 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0b519a-2bde-4367-8e30-7c4cf0fea0c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:10:39.922075 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0b519a-2bde-4367-8e30-7c4cf0fea0c1 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:10:39.931935 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:10:39.963550 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1138/2282] 10.222.0.59 () {60 vars in 1153 bytes} [Tue Oct 11 12:10:39 2022] POST /identity/v3/projects => generated 379 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 12:10:39.969096 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312ff1fb-89b1-4668-91f9-876782dbd95f admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:10:39.980026 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312ff1fb-89b1-4668-91f9-876782dbd95f admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:10:40.044055 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986cc251-d9a2-4c85-a53b-5248ce2901e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:10:40.044807 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986cc251-d9a2-4c85-a53b-5248ce2901e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:10:40.046269 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986cc251-d9a2-4c85-a53b-5248ce2901e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:10:40.046269 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986cc251-d9a2-4c85-a53b-5248ce2901e2 admin admin] PATH_INFO: `/v3/users` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:10:40.046269 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986cc251-d9a2-4c85-a53b-5248ce2901e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:10:40.049026 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986cc251-d9a2-4c85-a53b-5248ce2901e2 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:10:40.079141 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1145/2283] 10.222.0.59 () {60 vars in 1144 bytes} [Tue Oct 11 12:10:39 2022] POST /identity/v3/users => generated 290 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:10:40.083628 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0b519a-2bde-4367-8e30-7c4cf0fea0c1 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:10:40.097409 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0b519a-2bde-4367-8e30-7c4cf0fea0c1 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:10:40.103201 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037be7d1-5f04-4a3d-94c6-58b2cf82d91e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:10:40.103805 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037be7d1-5f04-4a3d-94c6-58b2cf82d91e admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:10:40.104081 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037be7d1-5f04-4a3d-94c6-58b2cf82d91e admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:10:40.104346 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037be7d1-5f04-4a3d-94c6-58b2cf82d91e admin admin] PATH_INFO: `/v3/roles` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:10:40.104985 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037be7d1-5f04-4a3d-94c6-58b2cf82d91e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:10:40.106589 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037be7d1-5f04-4a3d-94c6-58b2cf82d91e admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:10:40.117347 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1139/2284] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:10:40 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:10:40.121653 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986cc251-d9a2-4c85-a53b-5248ce2901e2 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:10:40.131888 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986cc251-d9a2-4c85-a53b-5248ce2901e2 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:10:40.136757 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9db55c-1ec2-4f60-8530-0b2248852116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:10:40.137354 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9db55c-1ec2-4f60-8530-0b2248852116 admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:10:40.137633 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9db55c-1ec2-4f60-8530-0b2248852116 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:10:40.137908 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9db55c-1ec2-4f60-8530-0b2248852116 admin admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:10:40.138585 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9db55c-1ec2-4f60-8530-0b2248852116 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:10:40.140324 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9db55c-1ec2-4f60-8530-0b2248852116 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:10:40.151583 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1146/2285] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:10:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:10:40.156530 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037be7d1-5f04-4a3d-94c6-58b2cf82d91e admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:10:40.172623 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037be7d1-5f04-4a3d-94c6-58b2cf82d91e admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:10:40.178729 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ed2636-b5d1-4479-8bdd-de30288ace24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:10:40.181861 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ed2636-b5d1-4479-8bdd-de30288ace24 admin admin] REQUEST_METHOD: `PUT` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:10:40.181861 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ed2636-b5d1-4479-8bdd-de30288ace24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:10:40.181861 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ed2636-b5d1-4479-8bdd-de30288ace24 admin admin] PATH_INFO: `/v3/projects/8a25c444677c48f199857cceee3341da/users/17ad5ea58a194f129ba616a882479fa5/roles/de2d2bd26fff4748a79990f97a107164` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:10:40.202815 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ed2636-b5d1-4479-8bdd-de30288ace24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a25c444677c48f199857cceee3341da, user_id=17ad5ea58a194f129ba616a882479fa5, role_id=de2d2bd26fff4748a79990f97a107164)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:10:40.205500 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ed2636-b5d1-4479-8bdd-de30288ace24 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:10:40.226416 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1140/2286] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 12:10:40 2022] PUT /identity/v3/projects/8a25c444677c48f199857cceee3341da/users/17ad5ea58a194f129ba616a882479fa5/roles/de2d2bd26fff4748a79990f97a107164 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:10:40.240860 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43520d76-3aa6-485e-9918-a3ce1bc81ecf None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:10:40.241359 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43520d76-3aa6-485e-9918-a3ce1bc81ecf None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:10:40.241483 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43520d76-3aa6-485e-9918-a3ce1bc81ecf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:10:40.271235 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-43520d76-3aa6-485e-9918-a3ce1bc81ecf None None] MFA Rules not processed for user `17ad5ea58a194f129ba616a882479fa5`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:10:40.272465 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-43520d76-3aa6-485e-9918-a3ce1bc81ecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:10:40.339358 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1147/2287] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:10:40 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:10:40.344048 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ed2636-b5d1-4479-8bdd-de30288ace24 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:10:40.412848 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ed2636-b5d1-4479-8bdd-de30288ace24 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:10:40.417971 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05d627e-c181-4906-b40b-7fa8f56f2b24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:10:40.418644 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05d627e-c181-4906-b40b-7fa8f56f2b24 admin admin] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:10:40.418905 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05d627e-c181-4906-b40b-7fa8f56f2b24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:10:40.419182 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05d627e-c181-4906-b40b-7fa8f56f2b24 admin admin] PATH_INFO: `/v3/projects` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:10:40.419970 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05d627e-c181-4906-b40b-7fa8f56f2b24 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:10:40.422081 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05d627e-c181-4906-b40b-7fa8f56f2b24 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:10:40.460207 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1141/2288] 10.222.0.59 () {60 vars in 1153 bytes} [Tue Oct 11 12:10:40 2022] POST /identity/v3/projects => generated 381 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:10:40.467697 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43520d76-3aa6-485e-9918-a3ce1bc81ecf None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:10:40.482676 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43520d76-3aa6-485e-9918-a3ce1bc81ecf None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:10:40.535601 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:10:40.556524 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0103ec73-1a50-4e96-b050-2169fe6a4784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:10:40.557924 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0103ec73-1a50-4e96-b050-2169fe6a4784 admin admin] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:10:40.558623 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0103ec73-1a50-4e96-b050-2169fe6a4784 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:10:40.558798 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0103ec73-1a50-4e96-b050-2169fe6a4784 admin admin] PATH_INFO: `/v3/users` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:10:40.559661 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0103ec73-1a50-4e96-b050-2169fe6a4784 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:10:40.562129 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0103ec73-1a50-4e96-b050-2169fe6a4784 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:10:40.590017 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1148/2289] 10.222.0.59 () {60 vars in 1144 bytes} [Tue Oct 11 12:10:40 2022] POST /identity/v3/users => generated 297 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:10:40.591278 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05d627e-c181-4906-b40b-7fa8f56f2b24 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:10:40.602486 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05d627e-c181-4906-b40b-7fa8f56f2b24 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:10:40.608888 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fecd37-490f-4f03-bc7e-710095eb749b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:10:40.609564 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fecd37-490f-4f03-bc7e-710095eb749b admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:10:40.609849 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fecd37-490f-4f03-bc7e-710095eb749b admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:10:40.610121 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fecd37-490f-4f03-bc7e-710095eb749b admin admin] PATH_INFO: `/v3/roles` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:10:40.610931 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fecd37-490f-4f03-bc7e-710095eb749b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:10:40.614405 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fecd37-490f-4f03-bc7e-710095eb749b admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:10:40.627417 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1142/2290] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:10:40 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 11 12:10:40.631467 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0103ec73-1a50-4e96-b050-2169fe6a4784 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:10:40.642110 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0103ec73-1a50-4e96-b050-2169fe6a4784 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:10:40.647084 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894f432b-5892-49b8-a691-220c563a6634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:10:40.647741 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894f432b-5892-49b8-a691-220c563a6634 admin admin] REQUEST_METHOD: `PUT` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:10:40.648026 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894f432b-5892-49b8-a691-220c563a6634 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:10:40.648301 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894f432b-5892-49b8-a691-220c563a6634 admin admin] PATH_INFO: `/v3/projects/f4b6eb5bfdee4145810dbe24d16a8ede/users/bc8d9906f7624aae85dd68e8099fc61c/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:10:40.664892 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894f432b-5892-49b8-a691-220c563a6634 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4b6eb5bfdee4145810dbe24d16a8ede, user_id=bc8d9906f7624aae85dd68e8099fc61c, role_id=5cf1f4bd2fef46ab9c6eff56d52f9f69)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:10:40.667642 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894f432b-5892-49b8-a691-220c563a6634 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:10:40.684476 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1149/2291] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 12:10:40 2022] PUT /identity/v3/projects/f4b6eb5bfdee4145810dbe24d16a8ede/users/bc8d9906f7624aae85dd68e8099fc61c/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:10:40.689386 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fecd37-490f-4f03-bc7e-710095eb749b admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:10:40.701384 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fecd37-490f-4f03-bc7e-710095eb749b admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:10:40.764525 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3731ff70-de47-4446-b961-795548bfc044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:10:40.765838 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3731ff70-de47-4446-b961-795548bfc044 admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:10:40.766675 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3731ff70-de47-4446-b961-795548bfc044 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:10:40.766940 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3731ff70-de47-4446-b961-795548bfc044 admin admin] PATH_INFO: `/v3/roles` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:10:40.767744 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3731ff70-de47-4446-b961-795548bfc044 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:10:40.769748 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3731ff70-de47-4446-b961-795548bfc044 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:10:40.779934 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1143/2292] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:10:40 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:10:40.786070 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894f432b-5892-49b8-a691-220c563a6634 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:10:40.797213 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894f432b-5892-49b8-a691-220c563a6634 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:10:40.801768 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74180bce-68ef-44b4-aed3-7c0b2e0f414f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:10:40.802417 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74180bce-68ef-44b4-aed3-7c0b2e0f414f admin admin] REQUEST_METHOD: `PUT` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:10:40.802727 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74180bce-68ef-44b4-aed3-7c0b2e0f414f admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:10:40.803004 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74180bce-68ef-44b4-aed3-7c0b2e0f414f admin admin] PATH_INFO: `/v3/projects/f4b6eb5bfdee4145810dbe24d16a8ede/users/bc8d9906f7624aae85dd68e8099fc61c/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:10:40.805032 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74180bce-68ef-44b4-aed3-7c0b2e0f414f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4b6eb5bfdee4145810dbe24d16a8ede, user_id=bc8d9906f7624aae85dd68e8099fc61c, role_id=5cf1f4bd2fef46ab9c6eff56d52f9f69)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:10:40.807858 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74180bce-68ef-44b4-aed3-7c0b2e0f414f admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:10:40.820624 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1150/2293] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 12:10:40 2022] PUT /identity/v3/projects/f4b6eb5bfdee4145810dbe24d16a8ede/users/bc8d9906f7624aae85dd68e8099fc61c/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 12:10:40.831955 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1104e9b-408b-4bb5-81c9-9610b800f713 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:10:40.832361 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1104e9b-408b-4bb5-81c9-9610b800f713 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:10:40.832649 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1104e9b-408b-4bb5-81c9-9610b800f713 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:10:40.861211 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-b1104e9b-408b-4bb5-81c9-9610b800f713 None None] MFA Rules not processed for user `bc8d9906f7624aae85dd68e8099fc61c`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:10:40.862454 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-b1104e9b-408b-4bb5-81c9-9610b800f713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:10:40.868987 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:10:40.969169 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1144/2294] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:10:40 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 12:10:40.980696 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74180bce-68ef-44b4-aed3-7c0b2e0f414f admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:10:40.982418 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-74180bce-68ef-44b4-aed3-7c0b2e0f414f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:10:41.067395 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74180bce-68ef-44b4-aed3-7c0b2e0f414f admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:10:41.071963 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ef9ab7-fb53-4606-9fdc-d8543375fcfd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b1b651c73664d08accc6edf2f79bace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:10:41.072578 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ef9ab7-fb53-4606-9fdc-d8543375fcfd service cinder] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:10:41.072855 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ef9ab7-fb53-4606-9fdc-d8543375fcfd service cinder] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:10:41.073128 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ef9ab7-fb53-4606-9fdc-d8543375fcfd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:10:41.085707 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ef9ab7-fb53-4606-9fdc-d8543375fcfd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:10:41.087941 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ef9ab7-fb53-4606-9fdc-d8543375fcfd service cinder] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:10:41.103175 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1151/2295] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:10:40 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:10:41.508466 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1104e9b-408b-4bb5-81c9-9610b800f713 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:10:41.518775 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1104e9b-408b-4bb5-81c9-9610b800f713 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:10:41.522901 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de81fd72-925e-4297-b961-e1971e6e6a5d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b1b651c73664d08accc6edf2f79bace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:10:41.523266 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de81fd72-925e-4297-b961-e1971e6e6a5d service cinder] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:10:41.523551 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de81fd72-925e-4297-b961-e1971e6e6a5d service cinder] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:10:41.523823 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de81fd72-925e-4297-b961-e1971e6e6a5d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:10:41.594072 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de81fd72-925e-4297-b961-e1971e6e6a5d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:10:41.595585 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de81fd72-925e-4297-b961-e1971e6e6a5d service cinder] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:10:41.605086 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1145/2296] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:10:41 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:10:42.006413 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eca17a6-eafc-4798-aed7-6e315e299204 None None] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:10:42.007095 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eca17a6-eafc-4798-aed7-6e315e299204 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:10:42.007095 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eca17a6-eafc-4798-aed7-6e315e299204 None None] PATH_INFO: `/` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:10:42.008596 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1152/2297] 10.222.0.59 () {54 vars in 921 bytes} [Tue Oct 11 12:10:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 11 12:10:42.014938 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de0fa29-64f0-4e80-8ea0-c3f739708a83 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:10:42.015615 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de0fa29-64f0-4e80-8ea0-c3f739708a83 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:10:42.016461 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de0fa29-64f0-4e80-8ea0-c3f739708a83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:10:42.069497 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-6de0fa29-64f0-4e80-8ea0-c3f739708a83 None None] MFA Rules not processed for user `2e563fe174db4cc19ae6bf8162cc3d23`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:10:42.073630 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-6de0fa29-64f0-4e80-8ea0-c3f739708a83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:10:42.137426 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1146/2298] 10.222.0.59 () {58 vars in 1018 bytes} [Tue Oct 11 12:10:42 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:10:42.150912 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eca17a6-eafc-4798-aed7-6e315e299204 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:10:42.151862 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-6eca17a6-eafc-4798-aed7-6e315e299204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:10:42.165523 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eca17a6-eafc-4798-aed7-6e315e299204 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:10:42.169224 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab86ce9-d15f-48a2-9870-c1b1289a43e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e563fe174db4cc19ae6bf8162cc3d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:10:42.169827 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab86ce9-d15f-48a2-9870-c1b1289a43e1 service neutron] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:10:42.170105 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab86ce9-d15f-48a2-9870-c1b1289a43e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:10:42.170378 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab86ce9-d15f-48a2-9870-c1b1289a43e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:10:42.177521 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab86ce9-d15f-48a2-9870-c1b1289a43e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:10:42.178998 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab86ce9-d15f-48a2-9870-c1b1289a43e1 service neutron] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:10:42.188102 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1153/2299] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:10:42 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:10:42.287445 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa95a09-3fef-4033-a0a8-55e824264a54 None None] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:10:42.287703 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa95a09-3fef-4033-a0a8-55e824264a54 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:10:42.288392 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa95a09-3fef-4033-a0a8-55e824264a54 None None] PATH_INFO: `/` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:10:42.289824 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1147/2300] 10.222.0.59 () {54 vars in 921 bytes} [Tue Oct 11 12:10:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 12:10:42.296174 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5129a1ce-75cb-44a3-9967-5a78067870a7 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:10:42.296635 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5129a1ce-75cb-44a3-9967-5a78067870a7 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:10:42.297010 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5129a1ce-75cb-44a3-9967-5a78067870a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:10:42.324960 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-5129a1ce-75cb-44a3-9967-5a78067870a7 None None] MFA Rules not processed for user `bad5e1cb306b46818096940acc3810b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:10:42.326409 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-5129a1ce-75cb-44a3-9967-5a78067870a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:10:42.388134 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:10:42.403611 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1154/2301] 10.222.0.59 () {58 vars in 1018 bytes} [Tue Oct 11 12:10:42 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:10:42.416601 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa95a09-3fef-4033-a0a8-55e824264a54 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:10:42.418938 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-7fa95a09-3fef-4033-a0a8-55e824264a54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:10:42.433282 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa95a09-3fef-4033-a0a8-55e824264a54 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:10:42.437232 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2aca9d-18bb-4977-a3dd-03ee2b0aaec4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e563fe174db4cc19ae6bf8162cc3d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:10:42.437492 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2aca9d-18bb-4977-a3dd-03ee2b0aaec4 service neutron] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:10:42.437766 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2aca9d-18bb-4977-a3dd-03ee2b0aaec4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:10:42.438033 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2aca9d-18bb-4977-a3dd-03ee2b0aaec4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:10:42.448102 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2aca9d-18bb-4977-a3dd-03ee2b0aaec4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:10:42.449517 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2aca9d-18bb-4977-a3dd-03ee2b0aaec4 service neutron] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:10:42.460238 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1148/2302] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:10:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:10:52.396839 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=2 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:10:52.455659 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=90 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:10:52.691311 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5129a1ce-75cb-44a3-9967-5a78067870a7 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:10:52.757138 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5129a1ce-75cb-44a3-9967-5a78067870a7 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:10:52.762024 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2496db-c8d5-4b68-992d-ca1c04c68188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:10:52.762696 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2496db-c8d5-4b68-992d-ca1c04c68188 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:10:52.763535 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2496db-c8d5-4b68-992d-ca1c04c68188 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:10:52.763535 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2496db-c8d5-4b68-992d-ca1c04c68188 admin admin] PATH_INFO: `/v3/users/17ad5ea58a194f129ba616a882479fa5` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:10:52.764670 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2496db-c8d5-4b68-992d-ca1c04c68188 admin admin] RBAC: Authorizing `identity:delete_user(user_id=17ad5ea58a194f129ba616a882479fa5)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:10:52.767052 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2496db-c8d5-4b68-992d-ca1c04c68188 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:10:52.818866 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-1c2496db-c8d5-4b68-992d-ca1c04c68188 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '17ad5ea58a194f129ba616a882479fa5'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:10:52.830729 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-1c2496db-c8d5-4b68-992d-ca1c04c68188 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '17ad5ea58a194f129ba616a882479fa5'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:10:52.831446 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-1c2496db-c8d5-4b68-992d-ca1c04c68188 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '17ad5ea58a194f129ba616a882479fa5'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:10:52.839454 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-1c2496db-c8d5-4b68-992d-ca1c04c68188 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '17ad5ea58a194f129ba616a882479fa5'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:10:52.840286 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-1c2496db-c8d5-4b68-992d-ca1c04c68188 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '17ad5ea58a194f129ba616a882479fa5'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:10:52.852494 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1155/2303] 10.222.0.59 () {58 vars in 1224 bytes} [Tue Oct 11 12:10:52 2022] DELETE /identity/v3/users/17ad5ea58a194f129ba616a882479fa5 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:10:52.857220 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc4f1d0-9962-41cb-acda-4474ec892078 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:10:52.857498 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc4f1d0-9962-41cb-acda-4474ec892078 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:10:52.857776 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc4f1d0-9962-41cb-acda-4474ec892078 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:10:52.877689 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-2cc4f1d0-9962-41cb-acda-4474ec892078 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:10:52.878952 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-2cc4f1d0-9962-41cb-acda-4474ec892078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:10:52.943747 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1149/2304] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:10:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:10:52.953400 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2496db-c8d5-4b68-992d-ca1c04c68188 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:10:52.954894 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-1c2496db-c8d5-4b68-992d-ca1c04c68188 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:10:53.023922 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2496db-c8d5-4b68-992d-ca1c04c68188 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:10:53.027713 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba461c83-a403-459a-95b5-fdefd7abf349 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e563fe174db4cc19ae6bf8162cc3d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:10:53.028330 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba461c83-a403-459a-95b5-fdefd7abf349 service neutron] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:10:53.028609 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba461c83-a403-459a-95b5-fdefd7abf349 service neutron] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:10:53.028879 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba461c83-a403-459a-95b5-fdefd7abf349 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:10:53.038698 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba461c83-a403-459a-95b5-fdefd7abf349 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:10:53.040151 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba461c83-a403-459a-95b5-fdefd7abf349 service neutron] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:10:53.050188 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1156/2305] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:10:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:10:53.332219 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc4f1d0-9962-41cb-acda-4474ec892078 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:10:53.333843 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-2cc4f1d0-9962-41cb-acda-4474ec892078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:10:53.347146 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc4f1d0-9962-41cb-acda-4474ec892078 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:10:53.352075 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13faa048-e316-49d1-a6f9-5a7a34035315 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:10:53.352729 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13faa048-e316-49d1-a6f9-5a7a34035315 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:10:53.353004 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13faa048-e316-49d1-a6f9-5a7a34035315 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:10:53.353276 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13faa048-e316-49d1-a6f9-5a7a34035315 admin admin] PATH_INFO: `/v3/projects/8a25c444677c48f199857cceee3341da` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:10:53.354402 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13faa048-e316-49d1-a6f9-5a7a34035315 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a25c444677c48f199857cceee3341da)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:10:53.356378 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13faa048-e316-49d1-a6f9-5a7a34035315 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:10:53.413170 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-13faa048-e316-49d1-a6f9-5a7a34035315 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a25c444677c48f199857cceee3341da', 'request_id': 'req-13faa048-e316-49d1-a6f9-5a7a34035315'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:10:53.419652 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-13faa048-e316-49d1-a6f9-5a7a34035315 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a25c444677c48f199857cceee3341da', 'request_id': 'req-13faa048-e316-49d1-a6f9-5a7a34035315'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:10:53.427412 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-13faa048-e316-49d1-a6f9-5a7a34035315 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a25c444677c48f199857cceee3341da', 'request_id': 'req-13faa048-e316-49d1-a6f9-5a7a34035315'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:10:53.439500 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-13faa048-e316-49d1-a6f9-5a7a34035315 admin admin] The token cache is being invalidate because project 8a25c444677c48f199857cceee3341da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 12:10:53.439714 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-13faa048-e316-49d1-a6f9-5a7a34035315 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:10:53.440473 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-13faa048-e316-49d1-a6f9-5a7a34035315 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:10:53.442065 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1150/2306] 10.222.0.59 () {58 vars in 1233 bytes} [Tue Oct 11 12:10:53 2022] DELETE /identity/v3/projects/8a25c444677c48f199857cceee3341da => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:10:53.446419 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba461c83-a403-459a-95b5-fdefd7abf349 service neutron] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:10:53.456103 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba461c83-a403-459a-95b5-fdefd7abf349 service neutron] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:10:53.476677 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:10:53.522606 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42218a3a-5680-4bd5-8d43-eb8dd4e1dec1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:10:53.523258 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42218a3a-5680-4bd5-8d43-eb8dd4e1dec1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:10:53.523537 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42218a3a-5680-4bd5-8d43-eb8dd4e1dec1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:10:53.523839 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42218a3a-5680-4bd5-8d43-eb8dd4e1dec1 admin admin] PATH_INFO: `/v3/users/bc8d9906f7624aae85dd68e8099fc61c` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:10:53.525020 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42218a3a-5680-4bd5-8d43-eb8dd4e1dec1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc8d9906f7624aae85dd68e8099fc61c)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:10:53.526994 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42218a3a-5680-4bd5-8d43-eb8dd4e1dec1 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:10:53.578190 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-42218a3a-5680-4bd5-8d43-eb8dd4e1dec1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc8d9906f7624aae85dd68e8099fc61c'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:10:53.590312 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-42218a3a-5680-4bd5-8d43-eb8dd4e1dec1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc8d9906f7624aae85dd68e8099fc61c'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:10:53.591038 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-42218a3a-5680-4bd5-8d43-eb8dd4e1dec1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc8d9906f7624aae85dd68e8099fc61c'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:10:53.600379 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-42218a3a-5680-4bd5-8d43-eb8dd4e1dec1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc8d9906f7624aae85dd68e8099fc61c'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:10:53.601059 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-42218a3a-5680-4bd5-8d43-eb8dd4e1dec1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc8d9906f7624aae85dd68e8099fc61c'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:10:53.613739 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1157/2307] 10.222.0.59 () {58 vars in 1224 bytes} [Tue Oct 11 12:10:53 2022] DELETE /identity/v3/users/bc8d9906f7624aae85dd68e8099fc61c => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:10:53.956685 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13faa048-e316-49d1-a6f9-5a7a34035315 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:10:53.961123 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-13faa048-e316-49d1-a6f9-5a7a34035315 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:10:53.976665 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13faa048-e316-49d1-a6f9-5a7a34035315 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:10:54.046012 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f9238c-1cb4-462f-a3b3-c4571681ecf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:10:54.046677 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f9238c-1cb4-462f-a3b3-c4571681ecf0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:10:54.046958 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f9238c-1cb4-462f-a3b3-c4571681ecf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:10:54.047227 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f9238c-1cb4-462f-a3b3-c4571681ecf0 admin admin] PATH_INFO: `/v3/projects/f4b6eb5bfdee4145810dbe24d16a8ede` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:10:54.048337 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f9238c-1cb4-462f-a3b3-c4571681ecf0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4b6eb5bfdee4145810dbe24d16a8ede)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:10:54.050233 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f9238c-1cb4-462f-a3b3-c4571681ecf0 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:10:54.083645 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:10:54.103456 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-44f9238c-1cb4-462f-a3b3-c4571681ecf0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4b6eb5bfdee4145810dbe24d16a8ede', 'request_id': 'req-44f9238c-1cb4-462f-a3b3-c4571681ecf0'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:10:54.109794 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-44f9238c-1cb4-462f-a3b3-c4571681ecf0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4b6eb5bfdee4145810dbe24d16a8ede', 'request_id': 'req-44f9238c-1cb4-462f-a3b3-c4571681ecf0'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:10:54.120285 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-44f9238c-1cb4-462f-a3b3-c4571681ecf0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4b6eb5bfdee4145810dbe24d16a8ede', 'request_id': 'req-44f9238c-1cb4-462f-a3b3-c4571681ecf0'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:10:54.132502 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-44f9238c-1cb4-462f-a3b3-c4571681ecf0 admin admin] The token cache is being invalidate because project f4b6eb5bfdee4145810dbe24d16a8ede was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 12:10:54.132797 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-44f9238c-1cb4-462f-a3b3-c4571681ecf0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:10:54.133520 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-44f9238c-1cb4-462f-a3b3-c4571681ecf0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:10:54.135186 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1151/2308] 10.222.0.59 () {58 vars in 1233 bytes} [Tue Oct 11 12:10:53 2022] DELETE /identity/v3/projects/f4b6eb5bfdee4145810dbe24d16a8ede => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:10:54.142656 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42249c6e-4dc4-4f0b-9d7c-fef2045cf759 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:10:54.142969 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42249c6e-4dc4-4f0b-9d7c-fef2045cf759 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:10:54.143259 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42249c6e-4dc4-4f0b-9d7c-fef2045cf759 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:10:54.166053 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-42249c6e-4dc4-4f0b-9d7c-fef2045cf759 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:10:54.167278 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-42249c6e-4dc4-4f0b-9d7c-fef2045cf759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:10:54.232064 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1158/2309] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:10:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:10:54.240214 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce4cadf-677d-42c3-998d-54fc53afc233 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:10:54.241222 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce4cadf-677d-42c3-998d-54fc53afc233 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:10:54.241222 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce4cadf-677d-42c3-998d-54fc53afc233 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:10:54.263806 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-4ce4cadf-677d-42c3-998d-54fc53afc233 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:10:54.265029 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-4ce4cadf-677d-42c3-998d-54fc53afc233 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:10:54.277033 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1152/2310] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:10:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:10:54.278342 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42249c6e-4dc4-4f0b-9d7c-fef2045cf759 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:10:54.289605 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42249c6e-4dc4-4f0b-9d7c-fef2045cf759 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:10:54.296064 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f593a9f4-cdc4-493f-84e5-7d6f8bdec1ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:10:54.296816 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f593a9f4-cdc4-493f-84e5-7d6f8bdec1ae admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:10:54.297218 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f593a9f4-cdc4-493f-84e5-7d6f8bdec1ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:10:54.297512 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f593a9f4-cdc4-493f-84e5-7d6f8bdec1ae admin admin] PATH_INFO: `/v3/domains` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:10:54.298227 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f593a9f4-cdc4-493f-84e5-7d6f8bdec1ae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 12:10:54.298615 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f593a9f4-cdc4-493f-84e5-7d6f8bdec1ae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:10:54.301800 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f593a9f4-cdc4-493f-84e5-7d6f8bdec1ae admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:10:54.313170 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1159/2311] 10.222.0.59 () {58 vars in 1153 bytes} [Tue Oct 11 12:10:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 12:10:54.317677 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce4cadf-677d-42c3-998d-54fc53afc233 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:10:54.327680 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce4cadf-677d-42c3-998d-54fc53afc233 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:10:54.333428 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19623e22-a22f-4886-9818-e079beccafe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:10:54.334061 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19623e22-a22f-4886-9818-e079beccafe4 admin admin] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:10:54.334342 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19623e22-a22f-4886-9818-e079beccafe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:10:54.334657 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19623e22-a22f-4886-9818-e079beccafe4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:10:54.335340 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19623e22-a22f-4886-9818-e079beccafe4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:10:54.337244 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19623e22-a22f-4886-9818-e079beccafe4 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:10:54.368191 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1153/2312] 10.222.0.59 () {60 vars in 1153 bytes} [Tue Oct 11 12:10:54 2022] POST /identity/v3/projects => generated 385 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:10:54.372514 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f593a9f4-cdc4-493f-84e5-7d6f8bdec1ae admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:10:54.382443 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f593a9f4-cdc4-493f-84e5-7d6f8bdec1ae admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:10:54.409658 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:10:54.445013 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5be95a8-e029-4f3d-b179-9de948dfa4d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:10:54.445649 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5be95a8-e029-4f3d-b179-9de948dfa4d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:10:54.445921 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5be95a8-e029-4f3d-b179-9de948dfa4d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:10:54.446194 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5be95a8-e029-4f3d-b179-9de948dfa4d9 admin admin] PATH_INFO: `/v3/users` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:10:54.446862 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5be95a8-e029-4f3d-b179-9de948dfa4d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:10:54.448793 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5be95a8-e029-4f3d-b179-9de948dfa4d9 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:10:54.472478 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1160/2313] 10.222.0.59 () {60 vars in 1144 bytes} [Tue Oct 11 12:10:54 2022] POST /identity/v3/users => generated 293 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 12:10:54.477004 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19623e22-a22f-4886-9818-e079beccafe4 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:10:54.487695 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19623e22-a22f-4886-9818-e079beccafe4 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:10:54.492815 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e07ea6-7b51-4380-8fb9-51799714e21b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:10:54.493431 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e07ea6-7b51-4380-8fb9-51799714e21b admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:10:54.493717 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e07ea6-7b51-4380-8fb9-51799714e21b admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:10:54.493986 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e07ea6-7b51-4380-8fb9-51799714e21b admin admin] PATH_INFO: `/v3/roles` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:10:54.494663 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e07ea6-7b51-4380-8fb9-51799714e21b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:10:54.496300 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e07ea6-7b51-4380-8fb9-51799714e21b admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:10:54.507542 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1154/2314] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:10:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:10:54.511945 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5be95a8-e029-4f3d-b179-9de948dfa4d9 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:10:54.523298 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5be95a8-e029-4f3d-b179-9de948dfa4d9 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:10:54.529237 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fb8c4f-4777-46e6-85ce-9849f8efa849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:10:54.529843 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fb8c4f-4777-46e6-85ce-9849f8efa849 admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:10:54.530117 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fb8c4f-4777-46e6-85ce-9849f8efa849 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:10:54.530382 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fb8c4f-4777-46e6-85ce-9849f8efa849 admin admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:10:54.531086 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fb8c4f-4777-46e6-85ce-9849f8efa849 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:10:54.532709 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fb8c4f-4777-46e6-85ce-9849f8efa849 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:10:54.543286 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1161/2315] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:10:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:10:54.547895 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e07ea6-7b51-4380-8fb9-51799714e21b admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:10:54.559399 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e07ea6-7b51-4380-8fb9-51799714e21b admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:10:54.564804 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242920a8-41d1-45d5-907a-ee77f9582acb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:10:54.565456 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242920a8-41d1-45d5-907a-ee77f9582acb admin admin] REQUEST_METHOD: `PUT` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:10:54.565724 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242920a8-41d1-45d5-907a-ee77f9582acb admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:10:54.566007 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242920a8-41d1-45d5-907a-ee77f9582acb admin admin] PATH_INFO: `/v3/projects/9d0239032311465da4c4793c9a6f1375/users/232871e0878b42a0b6c4d74d64e062a7/roles/de2d2bd26fff4748a79990f97a107164` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:10:54.582655 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242920a8-41d1-45d5-907a-ee77f9582acb admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d0239032311465da4c4793c9a6f1375, user_id=232871e0878b42a0b6c4d74d64e062a7, role_id=de2d2bd26fff4748a79990f97a107164)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:10:54.585398 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242920a8-41d1-45d5-907a-ee77f9582acb admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:10:54.600788 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1155/2316] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 12:10:54 2022] PUT /identity/v3/projects/9d0239032311465da4c4793c9a6f1375/users/232871e0878b42a0b6c4d74d64e062a7/roles/de2d2bd26fff4748a79990f97a107164 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:10:54.613676 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf47ffa-8262-4822-8913-10f54880918f None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:10:54.614050 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf47ffa-8262-4822-8913-10f54880918f None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:10:54.614329 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf47ffa-8262-4822-8913-10f54880918f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:10:54.648880 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-1bf47ffa-8262-4822-8913-10f54880918f None None] MFA Rules not processed for user `232871e0878b42a0b6c4d74d64e062a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:10:54.650126 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-1bf47ffa-8262-4822-8913-10f54880918f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:10:54.719038 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1162/2317] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:10:54 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:10:54.726234 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242920a8-41d1-45d5-907a-ee77f9582acb admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:10:54.790286 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242920a8-41d1-45d5-907a-ee77f9582acb admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:10:54.795028 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcd3685-ab64-488e-b1eb-9f826f7cf86a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:10:54.795630 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcd3685-ab64-488e-b1eb-9f826f7cf86a admin admin] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:10:54.796058 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcd3685-ab64-488e-b1eb-9f826f7cf86a admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:10:54.796192 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcd3685-ab64-488e-b1eb-9f826f7cf86a admin admin] PATH_INFO: `/v3/projects` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:10:54.796797 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcd3685-ab64-488e-b1eb-9f826f7cf86a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:10:54.798732 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcd3685-ab64-488e-b1eb-9f826f7cf86a admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:10:54.828320 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1156/2318] 10.222.0.59 () {60 vars in 1153 bytes} [Tue Oct 11 12:10:54 2022] POST /identity/v3/projects => generated 385 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:10:54.832539 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf47ffa-8262-4822-8913-10f54880918f None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:10:54.845324 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf47ffa-8262-4822-8913-10f54880918f None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:10:54.905509 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b501bc-66ad-4a1e-a499-b6c7df8daca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:10:54.906400 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b501bc-66ad-4a1e-a499-b6c7df8daca7 admin admin] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:10:54.906772 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b501bc-66ad-4a1e-a499-b6c7df8daca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:10:54.907049 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b501bc-66ad-4a1e-a499-b6c7df8daca7 admin admin] PATH_INFO: `/v3/users` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:10:54.907951 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b501bc-66ad-4a1e-a499-b6c7df8daca7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:10:54.910902 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b501bc-66ad-4a1e-a499-b6c7df8daca7 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:10:54.934730 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1163/2319] 10.222.0.59 () {60 vars in 1144 bytes} [Tue Oct 11 12:10:54 2022] POST /identity/v3/users => generated 299 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:10:54.939010 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcd3685-ab64-488e-b1eb-9f826f7cf86a admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:10:54.950146 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcd3685-ab64-488e-b1eb-9f826f7cf86a admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:10:54.955628 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f62ea4-cb4d-48b5-9fb0-4a2a0cfbac95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:10:54.956243 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f62ea4-cb4d-48b5-9fb0-4a2a0cfbac95 admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:10:54.956521 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f62ea4-cb4d-48b5-9fb0-4a2a0cfbac95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:10:54.956791 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f62ea4-cb4d-48b5-9fb0-4a2a0cfbac95 admin admin] PATH_INFO: `/v3/roles` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:10:54.957437 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f62ea4-cb4d-48b5-9fb0-4a2a0cfbac95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:10:54.959223 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f62ea4-cb4d-48b5-9fb0-4a2a0cfbac95 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:10:54.972831 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1157/2320] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:10:54 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:10:54.977325 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b501bc-66ad-4a1e-a499-b6c7df8daca7 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:10:54.988175 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b501bc-66ad-4a1e-a499-b6c7df8daca7 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:10:54.994268 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a766809b-1219-4745-9c0b-7d50af4c68ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:10:54.994974 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a766809b-1219-4745-9c0b-7d50af4c68ad admin admin] REQUEST_METHOD: `PUT` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:10:54.995247 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a766809b-1219-4745-9c0b-7d50af4c68ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:10:54.995512 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a766809b-1219-4745-9c0b-7d50af4c68ad admin admin] PATH_INFO: `/v3/projects/2f3569fa093b48bf900b6827dc7c1cae/users/2f78a4dc28e849d2b331e80b8dcce322/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:10:55.013340 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a766809b-1219-4745-9c0b-7d50af4c68ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f3569fa093b48bf900b6827dc7c1cae, user_id=2f78a4dc28e849d2b331e80b8dcce322, role_id=5cf1f4bd2fef46ab9c6eff56d52f9f69)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:10:55.017633 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a766809b-1219-4745-9c0b-7d50af4c68ad admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:10:55.033791 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1164/2321] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 12:10:54 2022] PUT /identity/v3/projects/2f3569fa093b48bf900b6827dc7c1cae/users/2f78a4dc28e849d2b331e80b8dcce322/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:10:55.038329 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f62ea4-cb4d-48b5-9fb0-4a2a0cfbac95 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:10:55.049394 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f62ea4-cb4d-48b5-9fb0-4a2a0cfbac95 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:10:55.070199 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:10:55.111700 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62391707-b98e-4485-a452-b72f5da3e93b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:10:55.112410 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62391707-b98e-4485-a452-b72f5da3e93b admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:10:55.112754 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62391707-b98e-4485-a452-b72f5da3e93b admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:10:55.113027 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62391707-b98e-4485-a452-b72f5da3e93b admin admin] PATH_INFO: `/v3/roles` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:10:55.113671 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62391707-b98e-4485-a452-b72f5da3e93b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:10:55.115365 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62391707-b98e-4485-a452-b72f5da3e93b admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:10:55.133781 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1158/2322] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:10:55 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:10:55.138523 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a766809b-1219-4745-9c0b-7d50af4c68ad admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:10:55.144660 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:10:55.152049 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a766809b-1219-4745-9c0b-7d50af4c68ad admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:10:55.157357 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d508f166-f937-4eb1-b21d-e284d58438b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:10:55.158028 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d508f166-f937-4eb1-b21d-e284d58438b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:10:55.158303 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d508f166-f937-4eb1-b21d-e284d58438b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:10:55.158599 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d508f166-f937-4eb1-b21d-e284d58438b3 admin admin] PATH_INFO: `/v3/projects/2f3569fa093b48bf900b6827dc7c1cae/users/2f78a4dc28e849d2b331e80b8dcce322/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:10:55.160569 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d508f166-f937-4eb1-b21d-e284d58438b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f3569fa093b48bf900b6827dc7c1cae, user_id=2f78a4dc28e849d2b331e80b8dcce322, role_id=5cf1f4bd2fef46ab9c6eff56d52f9f69)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:10:55.163350 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d508f166-f937-4eb1-b21d-e284d58438b3 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:10:55.173859 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1165/2323] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 12:10:55 2022] PUT /identity/v3/projects/2f3569fa093b48bf900b6827dc7c1cae/users/2f78a4dc28e849d2b331e80b8dcce322/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:10:55.184687 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2b53d0-5600-4f6f-947a-b7917f9a5d7e None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:10:55.184978 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2b53d0-5600-4f6f-947a-b7917f9a5d7e None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:10:55.185276 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2b53d0-5600-4f6f-947a-b7917f9a5d7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:10:55.216867 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-af2b53d0-5600-4f6f-947a-b7917f9a5d7e None None] MFA Rules not processed for user `2f78a4dc28e849d2b331e80b8dcce322`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:10:55.218098 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-af2b53d0-5600-4f6f-947a-b7917f9a5d7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:10:55.292871 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1159/2324] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:10:55 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:10:55.304492 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d508f166-f937-4eb1-b21d-e284d58438b3 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:10:55.306546 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-d508f166-f937-4eb1-b21d-e284d58438b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:10:55.366325 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d508f166-f937-4eb1-b21d-e284d58438b3 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:10:55.369974 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc31c65f-a5ae-4cb9-afd2-58c4432f1a0e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b1b651c73664d08accc6edf2f79bace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:10:55.370737 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc31c65f-a5ae-4cb9-afd2-58c4432f1a0e service cinder] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:10:55.371021 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc31c65f-a5ae-4cb9-afd2-58c4432f1a0e service cinder] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:10:55.371348 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc31c65f-a5ae-4cb9-afd2-58c4432f1a0e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:10:55.428780 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc31c65f-a5ae-4cb9-afd2-58c4432f1a0e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:10:55.430264 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc31c65f-a5ae-4cb9-afd2-58c4432f1a0e service cinder] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:10:55.439331 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1166/2325] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:10:55 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:10:59.218067 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2b53d0-5600-4f6f-947a-b7917f9a5d7e None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:10:59.219715 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-af2b53d0-5600-4f6f-947a-b7917f9a5d7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:10:59.303264 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2b53d0-5600-4f6f-947a-b7917f9a5d7e None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:10:59.312019 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e09119-900f-4803-9c3b-e21e14a4ac13 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bad5e1cb306b46818096940acc3810b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:10:59.312908 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e09119-900f-4803-9c3b-e21e14a4ac13 service nova] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:10:59.313261 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e09119-900f-4803-9c3b-e21e14a4ac13 service nova] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:10:59.313595 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e09119-900f-4803-9c3b-e21e14a4ac13 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:10:59.315933 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-09e09119-900f-4803-9c3b-e21e14a4ac13 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:10:59.331388 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e09119-900f-4803-9c3b-e21e14a4ac13 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:10:59.333111 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e09119-900f-4803-9c3b-e21e14a4ac13 service nova] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:10:59.345246 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1160/2326] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:10:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:10:59.779559 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc31c65f-a5ae-4cb9-afd2-58c4432f1a0e service cinder] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:10:59.788620 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc31c65f-a5ae-4cb9-afd2-58c4432f1a0e service cinder] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:10:59.793614 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4497ecea-843f-43cb-b187-daae346c8b3d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b1b651c73664d08accc6edf2f79bace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:10:59.794240 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4497ecea-843f-43cb-b187-daae346c8b3d service cinder] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:10:59.794519 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4497ecea-843f-43cb-b187-daae346c8b3d service cinder] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:10:59.794841 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4497ecea-843f-43cb-b187-daae346c8b3d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:10:59.804195 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4497ecea-843f-43cb-b187-daae346c8b3d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:10:59.805647 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4497ecea-843f-43cb-b187-daae346c8b3d service cinder] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:10:59.816853 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1167/2327] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:10:59 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:11:04.862486 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e09119-900f-4803-9c3b-e21e14a4ac13 service nova] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:11:04.864078 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-09e09119-900f-4803-9c3b-e21e14a4ac13 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:11:04.878715 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4497ecea-843f-43cb-b187-daae346c8b3d service cinder] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:11:04.898845 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:11:04.922238 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e09119-900f-4803-9c3b-e21e14a4ac13 service nova] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:11:04.926602 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd44cd50-812e-4902-8f2e-5bb634938767 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e563fe174db4cc19ae6bf8162cc3d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:11:04.927380 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd44cd50-812e-4902-8f2e-5bb634938767 service neutron] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:11:04.928002 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd44cd50-812e-4902-8f2e-5bb634938767 service neutron] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:11:04.928281 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd44cd50-812e-4902-8f2e-5bb634938767 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:11:04.930247 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-bd44cd50-812e-4902-8f2e-5bb634938767 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:11:04.940873 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4497ecea-843f-43cb-b187-daae346c8b3d service cinder] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:11:04.943009 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd44cd50-812e-4902-8f2e-5bb634938767 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:11:04.944457 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd44cd50-812e-4902-8f2e-5bb634938767 service neutron] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:11:04.945565 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e89f82-92e2-4b71-946d-5529688c7fef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:11:04.946199 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e89f82-92e2-4b71-946d-5529688c7fef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:11:04.946504 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e89f82-92e2-4b71-946d-5529688c7fef admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:11:04.946808 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e89f82-92e2-4b71-946d-5529688c7fef admin admin] PATH_INFO: `/v3/users/232871e0878b42a0b6c4d74d64e062a7` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:11:04.947906 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e89f82-92e2-4b71-946d-5529688c7fef admin admin] RBAC: Authorizing `identity:delete_user(user_id=232871e0878b42a0b6c4d74d64e062a7)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:11:04.949957 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e89f82-92e2-4b71-946d-5529688c7fef admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:11:04.955182 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1161/2328] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:11:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:11:05.000402 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:11:05.014693 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-f9e89f82-92e2-4b71-946d-5529688c7fef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '232871e0878b42a0b6c4d74d64e062a7'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:11:05.027595 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-f9e89f82-92e2-4b71-946d-5529688c7fef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '232871e0878b42a0b6c4d74d64e062a7'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:11:05.028332 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-f9e89f82-92e2-4b71-946d-5529688c7fef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '232871e0878b42a0b6c4d74d64e062a7'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:11:05.037088 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-f9e89f82-92e2-4b71-946d-5529688c7fef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '232871e0878b42a0b6c4d74d64e062a7'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:11:05.037758 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-f9e89f82-92e2-4b71-946d-5529688c7fef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '232871e0878b42a0b6c4d74d64e062a7'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:11:05.052423 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1168/2329] 10.222.0.59 () {58 vars in 1224 bytes} [Tue Oct 11 12:11:04 2022] DELETE /identity/v3/users/232871e0878b42a0b6c4d74d64e062a7 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 12:11:05.057146 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80cb8d5-5070-409d-a13f-9ee8326c12d1 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:11:05.057432 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80cb8d5-5070-409d-a13f-9ee8326c12d1 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:11:05.057741 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80cb8d5-5070-409d-a13f-9ee8326c12d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:11:05.077052 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-d80cb8d5-5070-409d-a13f-9ee8326c12d1 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:11:05.078257 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-d80cb8d5-5070-409d-a13f-9ee8326c12d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:11:05.148874 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1162/2330] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:11:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:11:05.159867 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e89f82-92e2-4b71-946d-5529688c7fef admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:11:05.161456 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-f9e89f82-92e2-4b71-946d-5529688c7fef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:11:05.223461 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e89f82-92e2-4b71-946d-5529688c7fef admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:11:05.228304 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa11f20-564b-4e61-b62a-a9998cc755fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e563fe174db4cc19ae6bf8162cc3d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:11:05.228918 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa11f20-564b-4e61-b62a-a9998cc755fa service neutron] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:11:05.229182 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa11f20-564b-4e61-b62a-a9998cc755fa service neutron] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:11:05.229453 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa11f20-564b-4e61-b62a-a9998cc755fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:11:05.240097 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa11f20-564b-4e61-b62a-a9998cc755fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:11:05.241544 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa11f20-564b-4e61-b62a-a9998cc755fa service neutron] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:11:05.251539 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1169/2331] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:11:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:11:05.525121 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80cb8d5-5070-409d-a13f-9ee8326c12d1 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:11:05.526952 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-d80cb8d5-5070-409d-a13f-9ee8326c12d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:11:05.540585 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80cb8d5-5070-409d-a13f-9ee8326c12d1 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:11:05.546193 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ae41b2-9752-453d-bc10-5be0708155ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:11:05.546826 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ae41b2-9752-453d-bc10-5be0708155ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:11:05.547118 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ae41b2-9752-453d-bc10-5be0708155ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:11:05.547395 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ae41b2-9752-453d-bc10-5be0708155ed admin admin] PATH_INFO: `/v3/projects/9d0239032311465da4c4793c9a6f1375` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:11:05.548712 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ae41b2-9752-453d-bc10-5be0708155ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d0239032311465da4c4793c9a6f1375)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:11:05.550754 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ae41b2-9752-453d-bc10-5be0708155ed admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:11:05.604183 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-87ae41b2-9752-453d-bc10-5be0708155ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d0239032311465da4c4793c9a6f1375', 'request_id': 'req-87ae41b2-9752-453d-bc10-5be0708155ed'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:11:05.609897 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-87ae41b2-9752-453d-bc10-5be0708155ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d0239032311465da4c4793c9a6f1375', 'request_id': 'req-87ae41b2-9752-453d-bc10-5be0708155ed'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:11:05.618041 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-87ae41b2-9752-453d-bc10-5be0708155ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d0239032311465da4c4793c9a6f1375', 'request_id': 'req-87ae41b2-9752-453d-bc10-5be0708155ed'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:11:05.631498 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-87ae41b2-9752-453d-bc10-5be0708155ed admin admin] The token cache is being invalidate because project 9d0239032311465da4c4793c9a6f1375 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 12:11:05.631746 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-87ae41b2-9752-453d-bc10-5be0708155ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:11:05.632504 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-87ae41b2-9752-453d-bc10-5be0708155ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:11:05.634114 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1163/2332] 10.222.0.59 () {58 vars in 1233 bytes} [Tue Oct 11 12:11:05 2022] DELETE /identity/v3/projects/9d0239032311465da4c4793c9a6f1375 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:11:05.638811 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa11f20-564b-4e61-b62a-a9998cc755fa service neutron] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:11:05.649914 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa11f20-564b-4e61-b62a-a9998cc755fa service neutron] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:11:05.714115 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2380e1-3d3a-4ed4-a493-a0f2190fdc2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:11:05.714511 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2380e1-3d3a-4ed4-a493-a0f2190fdc2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:11:05.714815 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2380e1-3d3a-4ed4-a493-a0f2190fdc2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:11:05.715086 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2380e1-3d3a-4ed4-a493-a0f2190fdc2d admin admin] PATH_INFO: `/v3/users/2f78a4dc28e849d2b331e80b8dcce322` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:11:05.716252 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2380e1-3d3a-4ed4-a493-a0f2190fdc2d admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f78a4dc28e849d2b331e80b8dcce322)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:11:05.718185 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2380e1-3d3a-4ed4-a493-a0f2190fdc2d admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:11:05.772913 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-1b2380e1-3d3a-4ed4-a493-a0f2190fdc2d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f78a4dc28e849d2b331e80b8dcce322'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:11:05.779018 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:11:05.787964 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-1b2380e1-3d3a-4ed4-a493-a0f2190fdc2d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f78a4dc28e849d2b331e80b8dcce322'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:11:05.788969 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-1b2380e1-3d3a-4ed4-a493-a0f2190fdc2d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f78a4dc28e849d2b331e80b8dcce322'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:11:05.797139 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-1b2380e1-3d3a-4ed4-a493-a0f2190fdc2d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f78a4dc28e849d2b331e80b8dcce322'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:11:05.797749 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-1b2380e1-3d3a-4ed4-a493-a0f2190fdc2d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f78a4dc28e849d2b331e80b8dcce322'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:11:05.810331 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1170/2333] 10.222.0.59 () {58 vars in 1224 bytes} [Tue Oct 11 12:11:05 2022] DELETE /identity/v3/users/2f78a4dc28e849d2b331e80b8dcce322 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:11:06.109240 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ae41b2-9752-453d-bc10-5be0708155ed admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:11:06.110828 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-87ae41b2-9752-453d-bc10-5be0708155ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:11:06.143250 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ae41b2-9752-453d-bc10-5be0708155ed admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:11:06.209775 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:11:06.217878 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c23294e-791d-42a0-84a0-1434926750de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:11:06.219111 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c23294e-791d-42a0-84a0-1434926750de admin admin] REQUEST_METHOD: `DELETE` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:11:06.219712 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c23294e-791d-42a0-84a0-1434926750de admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:11:06.220390 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c23294e-791d-42a0-84a0-1434926750de admin admin] PATH_INFO: `/v3/projects/2f3569fa093b48bf900b6827dc7c1cae` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:11:06.221611 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c23294e-791d-42a0-84a0-1434926750de admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f3569fa093b48bf900b6827dc7c1cae)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:11:06.224203 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c23294e-791d-42a0-84a0-1434926750de admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:11:06.280390 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-7c23294e-791d-42a0-84a0-1434926750de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f3569fa093b48bf900b6827dc7c1cae', 'request_id': 'req-7c23294e-791d-42a0-84a0-1434926750de'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:11:06.286403 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-7c23294e-791d-42a0-84a0-1434926750de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f3569fa093b48bf900b6827dc7c1cae', 'request_id': 'req-7c23294e-791d-42a0-84a0-1434926750de'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:11:06.294713 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-7c23294e-791d-42a0-84a0-1434926750de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f3569fa093b48bf900b6827dc7c1cae', 'request_id': 'req-7c23294e-791d-42a0-84a0-1434926750de'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:11:06.308479 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-7c23294e-791d-42a0-84a0-1434926750de admin admin] The token cache is being invalidate because project 2f3569fa093b48bf900b6827dc7c1cae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 12:11:06.308479 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-7c23294e-791d-42a0-84a0-1434926750de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:11:06.309106 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-7c23294e-791d-42a0-84a0-1434926750de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:11:06.310700 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1164/2334] 10.222.0.59 () {58 vars in 1233 bytes} [Tue Oct 11 12:11:06 2022] DELETE /identity/v3/projects/2f3569fa093b48bf900b6827dc7c1cae => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:11:06.316782 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d476faca-8c22-4be6-b829-3fe6d5c9be04 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:11:06.317279 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d476faca-8c22-4be6-b829-3fe6d5c9be04 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:11:06.317451 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d476faca-8c22-4be6-b829-3fe6d5c9be04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:11:06.338492 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-d476faca-8c22-4be6-b829-3fe6d5c9be04 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:11:06.339864 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-d476faca-8c22-4be6-b829-3fe6d5c9be04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:11:06.413490 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1171/2335] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:11:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:11:06.415175 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c260e52-2fd7-4da2-a814-dbeb79b509bc None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:11:06.415463 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c260e52-2fd7-4da2-a814-dbeb79b509bc None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:11:06.415747 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c260e52-2fd7-4da2-a814-dbeb79b509bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:11:06.437252 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-1c260e52-2fd7-4da2-a814-dbeb79b509bc None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:11:06.438492 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-1c260e52-2fd7-4da2-a814-dbeb79b509bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:11:06.447172 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1165/2336] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:11:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:11:06.451217 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d476faca-8c22-4be6-b829-3fe6d5c9be04 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:11:06.463119 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d476faca-8c22-4be6-b829-3fe6d5c9be04 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:11:06.475263 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99cf650-71f5-4414-831a-bc0ffd4bb7c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:11:06.476483 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99cf650-71f5-4414-831a-bc0ffd4bb7c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:11:06.477238 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99cf650-71f5-4414-831a-bc0ffd4bb7c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:11:06.477590 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99cf650-71f5-4414-831a-bc0ffd4bb7c0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:11:06.478347 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99cf650-71f5-4414-831a-bc0ffd4bb7c0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 12:11:06.478738 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99cf650-71f5-4414-831a-bc0ffd4bb7c0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:11:06.481054 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99cf650-71f5-4414-831a-bc0ffd4bb7c0 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:11:06.493845 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1172/2337] 10.222.0.59 () {58 vars in 1153 bytes} [Tue Oct 11 12:11:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 12:11:06.498043 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c260e52-2fd7-4da2-a814-dbeb79b509bc None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:11:06.508433 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c260e52-2fd7-4da2-a814-dbeb79b509bc None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:11:06.514857 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb48a02-2428-4fac-8b34-da74c5abfbfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:11:06.516067 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb48a02-2428-4fac-8b34-da74c5abfbfc admin admin] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:11:06.516067 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb48a02-2428-4fac-8b34-da74c5abfbfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:11:06.516270 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb48a02-2428-4fac-8b34-da74c5abfbfc admin admin] PATH_INFO: `/v3/projects` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:11:06.517447 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb48a02-2428-4fac-8b34-da74c5abfbfc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:11:06.519674 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb48a02-2428-4fac-8b34-da74c5abfbfc admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:11:06.554466 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1166/2338] 10.222.0.59 () {60 vars in 1153 bytes} [Tue Oct 11 12:11:06 2022] POST /identity/v3/projects => generated 373 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:11:06.555643 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99cf650-71f5-4414-831a-bc0ffd4bb7c0 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:11:06.565854 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99cf650-71f5-4414-831a-bc0ffd4bb7c0 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:11:06.637342 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f77863a-b2ea-4cea-b70c-3344b22c6902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:11:06.638068 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f77863a-b2ea-4cea-b70c-3344b22c6902 admin admin] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:11:06.638351 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f77863a-b2ea-4cea-b70c-3344b22c6902 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:11:06.638653 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f77863a-b2ea-4cea-b70c-3344b22c6902 admin admin] PATH_INFO: `/v3/users` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:11:06.639647 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f77863a-b2ea-4cea-b70c-3344b22c6902 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:11:06.641597 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f77863a-b2ea-4cea-b70c-3344b22c6902 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:11:06.673526 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1173/2339] 10.222.0.59 () {60 vars in 1144 bytes} [Tue Oct 11 12:11:06 2022] POST /identity/v3/users => generated 287 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 12:11:06.674733 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb48a02-2428-4fac-8b34-da74c5abfbfc admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:11:06.685957 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb48a02-2428-4fac-8b34-da74c5abfbfc admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:11:06.692122 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef1e4b3-dd5d-4be8-8449-a8f7537c6474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:11:06.692725 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef1e4b3-dd5d-4be8-8449-a8f7537c6474 admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:11:06.693125 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef1e4b3-dd5d-4be8-8449-a8f7537c6474 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:11:06.693990 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef1e4b3-dd5d-4be8-8449-a8f7537c6474 admin admin] PATH_INFO: `/v3/roles` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:11:06.694699 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef1e4b3-dd5d-4be8-8449-a8f7537c6474 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:11:06.696429 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef1e4b3-dd5d-4be8-8449-a8f7537c6474 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:11:06.713725 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1167/2340] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:11:06 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 11 12:11:06.715773 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f77863a-b2ea-4cea-b70c-3344b22c6902 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:11:06.727572 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f77863a-b2ea-4cea-b70c-3344b22c6902 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:11:06.732341 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fabfab-075a-4176-8178-c5417c363baf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:11:06.732948 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fabfab-075a-4176-8178-c5417c363baf admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:11:06.733224 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fabfab-075a-4176-8178-c5417c363baf admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:11:06.733496 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fabfab-075a-4176-8178-c5417c363baf admin admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:11:06.734137 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fabfab-075a-4176-8178-c5417c363baf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:11:06.735810 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fabfab-075a-4176-8178-c5417c363baf admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:11:06.751774 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1174/2341] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:11:06 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:11:06.753736 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef1e4b3-dd5d-4be8-8449-a8f7537c6474 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:11:06.769003 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef1e4b3-dd5d-4be8-8449-a8f7537c6474 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:11:06.774308 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d22f1d-04fa-4c05-9695-dac51d04b629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:11:06.775301 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d22f1d-04fa-4c05-9695-dac51d04b629 admin admin] REQUEST_METHOD: `PUT` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:11:06.775767 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d22f1d-04fa-4c05-9695-dac51d04b629 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:11:06.776215 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d22f1d-04fa-4c05-9695-dac51d04b629 admin admin] PATH_INFO: `/v3/projects/dba120bd8056457cb511343c109c18b2/users/128dcace53d44c649e832ba7b72f6369/roles/de2d2bd26fff4748a79990f97a107164` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:11:06.793045 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d22f1d-04fa-4c05-9695-dac51d04b629 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dba120bd8056457cb511343c109c18b2, user_id=128dcace53d44c649e832ba7b72f6369, role_id=de2d2bd26fff4748a79990f97a107164)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:11:06.797801 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d22f1d-04fa-4c05-9695-dac51d04b629 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:11:06.834678 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1168/2342] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 12:11:06 2022] PUT /identity/v3/projects/dba120bd8056457cb511343c109c18b2/users/128dcace53d44c649e832ba7b72f6369/roles/de2d2bd26fff4748a79990f97a107164 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:11:06.839018 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd172cd-9ea2-4e4c-b478-afa44dcecb72 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:11:06.839542 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd172cd-9ea2-4e4c-b478-afa44dcecb72 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:11:06.840185 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd172cd-9ea2-4e4c-b478-afa44dcecb72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:11:06.883994 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-bcd172cd-9ea2-4e4c-b478-afa44dcecb72 None None] MFA Rules not processed for user `128dcace53d44c649e832ba7b72f6369`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:11:06.885563 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-bcd172cd-9ea2-4e4c-b478-afa44dcecb72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:11:06.914879 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:11:06.994183 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1175/2343] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:11:06 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:11:07.007409 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d22f1d-04fa-4c05-9695-dac51d04b629 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:11:07.010199 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-c0d22f1d-04fa-4c05-9695-dac51d04b629 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:11:07.078340 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d22f1d-04fa-4c05-9695-dac51d04b629 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:11:07.082170 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffece83b-0469-48f9-a432-56162d91fdf9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b1b651c73664d08accc6edf2f79bace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:11:07.082934 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffece83b-0469-48f9-a432-56162d91fdf9 service cinder] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:11:07.083215 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffece83b-0469-48f9-a432-56162d91fdf9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:11:07.083482 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffece83b-0469-48f9-a432-56162d91fdf9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:11:07.092422 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffece83b-0469-48f9-a432-56162d91fdf9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:11:07.093855 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffece83b-0469-48f9-a432-56162d91fdf9 service cinder] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:11:07.103357 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1169/2344] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:11:07 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:11:07.129105 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd172cd-9ea2-4e4c-b478-afa44dcecb72 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:11:07.208082 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd172cd-9ea2-4e4c-b478-afa44dcecb72 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:11:07.213631 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f249b3-6320-4785-a803-5833e81f9b7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:11:07.213960 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f249b3-6320-4785-a803-5833e81f9b7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:11:07.214074 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f249b3-6320-4785-a803-5833e81f9b7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:11:07.214457 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f249b3-6320-4785-a803-5833e81f9b7c admin admin] PATH_INFO: `/v3/users/128dcace53d44c649e832ba7b72f6369` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:11:07.216041 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f249b3-6320-4785-a803-5833e81f9b7c admin admin] RBAC: Authorizing `identity:delete_user(user_id=128dcace53d44c649e832ba7b72f6369)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:11:07.221597 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f249b3-6320-4785-a803-5833e81f9b7c admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:11:07.289793 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-67f249b3-6320-4785-a803-5833e81f9b7c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '128dcace53d44c649e832ba7b72f6369'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:11:07.304153 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-67f249b3-6320-4785-a803-5833e81f9b7c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '128dcace53d44c649e832ba7b72f6369'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:11:07.304957 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-67f249b3-6320-4785-a803-5833e81f9b7c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '128dcace53d44c649e832ba7b72f6369'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:11:07.313377 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-67f249b3-6320-4785-a803-5833e81f9b7c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '128dcace53d44c649e832ba7b72f6369'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:11:07.314068 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-67f249b3-6320-4785-a803-5833e81f9b7c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '128dcace53d44c649e832ba7b72f6369'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:11:07.327662 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1176/2345] 10.222.0.59 () {58 vars in 1224 bytes} [Tue Oct 11 12:11:07 2022] DELETE /identity/v3/users/128dcace53d44c649e832ba7b72f6369 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:11:07.332383 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e0c601-b258-49e6-9bfc-2f18dc7b59b8 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:11:07.332835 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e0c601-b258-49e6-9bfc-2f18dc7b59b8 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:11:07.332946 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e0c601-b258-49e6-9bfc-2f18dc7b59b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:11:07.353553 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-47e0c601-b258-49e6-9bfc-2f18dc7b59b8 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:11:07.355296 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-47e0c601-b258-49e6-9bfc-2f18dc7b59b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:11:07.377620 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:11:07.420104 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1170/2346] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:11:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:11:07.431247 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f249b3-6320-4785-a803-5833e81f9b7c admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:11:07.432842 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-67f249b3-6320-4785-a803-5833e81f9b7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:11:07.487431 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:11:07.493531 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f249b3-6320-4785-a803-5833e81f9b7c admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:11:07.499132 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d696a14f-8611-411c-b5c6-4103fa5f78bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e563fe174db4cc19ae6bf8162cc3d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:11:07.500081 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d696a14f-8611-411c-b5c6-4103fa5f78bf service neutron] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:11:07.500522 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d696a14f-8611-411c-b5c6-4103fa5f78bf service neutron] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:11:07.500981 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d696a14f-8611-411c-b5c6-4103fa5f78bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:11:07.511431 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d696a14f-8611-411c-b5c6-4103fa5f78bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:11:07.512942 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d696a14f-8611-411c-b5c6-4103fa5f78bf service neutron] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:11:07.523199 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1177/2347] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:11:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 11 12:11:07.841822 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e0c601-b258-49e6-9bfc-2f18dc7b59b8 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:11:07.843994 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-47e0c601-b258-49e6-9bfc-2f18dc7b59b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:11:07.856969 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e0c601-b258-49e6-9bfc-2f18dc7b59b8 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:11:07.867849 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1d764d-c5ca-47fa-884e-b1fc317f221f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:11:07.868609 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1d764d-c5ca-47fa-884e-b1fc317f221f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:11:07.870848 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1d764d-c5ca-47fa-884e-b1fc317f221f admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:11:07.871200 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1d764d-c5ca-47fa-884e-b1fc317f221f admin admin] PATH_INFO: `/v3/projects/dba120bd8056457cb511343c109c18b2` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:11:07.876067 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1d764d-c5ca-47fa-884e-b1fc317f221f admin admin] RBAC: Authorizing `identity:delete_project(project_id=dba120bd8056457cb511343c109c18b2)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:11:07.878036 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1d764d-c5ca-47fa-884e-b1fc317f221f admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:11:07.932675 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-9e1d764d-c5ca-47fa-884e-b1fc317f221f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dba120bd8056457cb511343c109c18b2', 'request_id': 'req-9e1d764d-c5ca-47fa-884e-b1fc317f221f'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:11:07.938444 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-9e1d764d-c5ca-47fa-884e-b1fc317f221f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dba120bd8056457cb511343c109c18b2', 'request_id': 'req-9e1d764d-c5ca-47fa-884e-b1fc317f221f'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:11:07.948111 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-9e1d764d-c5ca-47fa-884e-b1fc317f221f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dba120bd8056457cb511343c109c18b2', 'request_id': 'req-9e1d764d-c5ca-47fa-884e-b1fc317f221f'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:11:07.960367 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-9e1d764d-c5ca-47fa-884e-b1fc317f221f admin admin] The token cache is being invalidate because project dba120bd8056457cb511343c109c18b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 12:11:07.960727 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-9e1d764d-c5ca-47fa-884e-b1fc317f221f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:11:07.961323 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-9e1d764d-c5ca-47fa-884e-b1fc317f221f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:11:07.966516 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1171/2348] 10.222.0.59 () {58 vars in 1233 bytes} [Tue Oct 11 12:11:07 2022] DELETE /identity/v3/projects/dba120bd8056457cb511343c109c18b2 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:11:07.968434 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ab2d1c-76d4-462f-91b0-b2b21e424813 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:11:07.968726 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ab2d1c-76d4-462f-91b0-b2b21e424813 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:11:07.969005 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ab2d1c-76d4-462f-91b0-b2b21e424813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:11:07.991074 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-69ab2d1c-76d4-462f-91b0-b2b21e424813 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:11:07.992447 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-69ab2d1c-76d4-462f-91b0-b2b21e424813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:11:08.063351 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1178/2349] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:11:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:11:08.070855 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533645da-52da-4f3d-bf5b-6e6e41997ac9 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:11:08.071167 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533645da-52da-4f3d-bf5b-6e6e41997ac9 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:11:08.071445 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533645da-52da-4f3d-bf5b-6e6e41997ac9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:11:08.097045 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-533645da-52da-4f3d-bf5b-6e6e41997ac9 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:11:08.098292 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-533645da-52da-4f3d-bf5b-6e6e41997ac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:11:08.110881 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1172/2350] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:11:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 12:11:08.114874 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ab2d1c-76d4-462f-91b0-b2b21e424813 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:11:08.124973 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ab2d1c-76d4-462f-91b0-b2b21e424813 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:11:08.129546 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e4f57c-0a75-4795-bab9-230aba1645cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:11:08.130150 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e4f57c-0a75-4795-bab9-230aba1645cb admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:11:08.130430 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e4f57c-0a75-4795-bab9-230aba1645cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:11:08.133180 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e4f57c-0a75-4795-bab9-230aba1645cb admin admin] PATH_INFO: `/v3/domains` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:11:08.133180 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e4f57c-0a75-4795-bab9-230aba1645cb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 12:11:08.133180 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e4f57c-0a75-4795-bab9-230aba1645cb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:11:08.133445 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e4f57c-0a75-4795-bab9-230aba1645cb admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:11:08.146885 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1179/2351] 10.222.0.59 () {58 vars in 1153 bytes} [Tue Oct 11 12:11:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 12:11:08.150934 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533645da-52da-4f3d-bf5b-6e6e41997ac9 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:11:08.160600 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533645da-52da-4f3d-bf5b-6e6e41997ac9 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:11:08.165851 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2489c7b-ab11-467a-b00d-d24d7a0279c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:11:08.166196 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2489c7b-ab11-467a-b00d-d24d7a0279c7 admin admin] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:11:08.166459 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2489c7b-ab11-467a-b00d-d24d7a0279c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:11:08.166785 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2489c7b-ab11-467a-b00d-d24d7a0279c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:11:08.167597 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2489c7b-ab11-467a-b00d-d24d7a0279c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:11:08.169604 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2489c7b-ab11-467a-b00d-d24d7a0279c7 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:11:08.204284 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1173/2352] 10.222.0.59 () {60 vars in 1153 bytes} [Tue Oct 11 12:11:08 2022] POST /identity/v3/projects => generated 375 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:11:08.208414 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e4f57c-0a75-4795-bab9-230aba1645cb admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:11:08.220719 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e4f57c-0a75-4795-bab9-230aba1645cb admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:11:08.281474 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15c83cc-7e87-4f80-8d33-055af52c6850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:11:08.283722 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15c83cc-7e87-4f80-8d33-055af52c6850 admin admin] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:11:08.283722 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15c83cc-7e87-4f80-8d33-055af52c6850 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:11:08.283722 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15c83cc-7e87-4f80-8d33-055af52c6850 admin admin] PATH_INFO: `/v3/users` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:11:08.284158 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15c83cc-7e87-4f80-8d33-055af52c6850 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:11:08.286299 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15c83cc-7e87-4f80-8d33-055af52c6850 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:11:08.312256 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2489c7b-ab11-467a-b00d-d24d7a0279c7 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:11:08.314179 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-e2489c7b-ab11-467a-b00d-d24d7a0279c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:11:08.324434 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1180/2353] 10.222.0.59 () {60 vars in 1144 bytes} [Tue Oct 11 12:11:08 2022] POST /identity/v3/users => generated 288 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 12:11:08.328497 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15c83cc-7e87-4f80-8d33-055af52c6850 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:11:08.334678 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2489c7b-ab11-467a-b00d-d24d7a0279c7 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:11:08.340033 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cebcbc0-b834-4b74-8dc8-4eafc21ca2b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:11:08.340577 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15c83cc-7e87-4f80-8d33-055af52c6850 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:11:08.340705 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cebcbc0-b834-4b74-8dc8-4eafc21ca2b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:11:08.340949 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cebcbc0-b834-4b74-8dc8-4eafc21ca2b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:11:08.341395 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cebcbc0-b834-4b74-8dc8-4eafc21ca2b8 admin admin] PATH_INFO: `/v3/users/a78e40d4eb554ca49eeec530a7adf428` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:11:08.342535 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cebcbc0-b834-4b74-8dc8-4eafc21ca2b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a78e40d4eb554ca49eeec530a7adf428)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:11:08.344899 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cebcbc0-b834-4b74-8dc8-4eafc21ca2b8 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:11:08.345495 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c012fd-e13f-4805-9166-3aaf8acd0d8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:11:08.346113 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c012fd-e13f-4805-9166-3aaf8acd0d8a admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:11:08.346390 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c012fd-e13f-4805-9166-3aaf8acd0d8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:11:08.346724 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c012fd-e13f-4805-9166-3aaf8acd0d8a admin admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:11:08.347473 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c012fd-e13f-4805-9166-3aaf8acd0d8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:11:08.349218 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c012fd-e13f-4805-9166-3aaf8acd0d8a admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:11:08.361493 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1181/2354] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:11:08 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:11:08.365282 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c012fd-e13f-4805-9166-3aaf8acd0d8a admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:11:08.377559 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c012fd-e13f-4805-9166-3aaf8acd0d8a admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:11:08.383447 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5988176f-ffe5-4ca6-9343-546301a59fe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:11:08.384475 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5988176f-ffe5-4ca6-9343-546301a59fe4 admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:11:08.384793 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5988176f-ffe5-4ca6-9343-546301a59fe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:11:08.385028 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5988176f-ffe5-4ca6-9343-546301a59fe4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:11:08.385837 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5988176f-ffe5-4ca6-9343-546301a59fe4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:11:08.387667 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5988176f-ffe5-4ca6-9343-546301a59fe4 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:11:08.406846 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1182/2355] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:11:08 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:11:08.411815 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-5cebcbc0-b834-4b74-8dc8-4eafc21ca2b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a78e40d4eb554ca49eeec530a7adf428'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:11:08.415283 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5988176f-ffe5-4ca6-9343-546301a59fe4 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:11:08.436487 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5988176f-ffe5-4ca6-9343-546301a59fe4 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:11:08.437165 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-5cebcbc0-b834-4b74-8dc8-4eafc21ca2b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a78e40d4eb554ca49eeec530a7adf428'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:11:08.441425 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb39e1c-f1ea-4ca6-8285-94c6e0dd47f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:11:08.442102 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb39e1c-f1ea-4ca6-8285-94c6e0dd47f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:11:08.442380 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb39e1c-f1ea-4ca6-8285-94c6e0dd47f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:11:08.445836 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb39e1c-f1ea-4ca6-8285-94c6e0dd47f5 admin admin] PATH_INFO: `/v3/projects/6e42ea3c037b44b1a6dc2ca0a9ffdd6c/users/3434311c8240420896d6bfcb3428e515/roles/de2d2bd26fff4748a79990f97a107164` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:11:08.451141 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-5cebcbc0-b834-4b74-8dc8-4eafc21ca2b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a78e40d4eb554ca49eeec530a7adf428'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:11:08.451798 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-5cebcbc0-b834-4b74-8dc8-4eafc21ca2b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a78e40d4eb554ca49eeec530a7adf428'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:11:08.457265 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=80,keystone:DELETE=17,keystone:INSERT=3 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:11:08.462313 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb39e1c-f1ea-4ca6-8285-94c6e0dd47f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e42ea3c037b44b1a6dc2ca0a9ffdd6c, user_id=3434311c8240420896d6bfcb3428e515, role_id=de2d2bd26fff4748a79990f97a107164)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:11:08.462930 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-5cebcbc0-b834-4b74-8dc8-4eafc21ca2b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a78e40d4eb554ca49eeec530a7adf428'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:11:08.465433 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb39e1c-f1ea-4ca6-8285-94c6e0dd47f5 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:11:08.472329 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1174/2356] 10.222.0.59 () {58 vars in 1224 bytes} [Tue Oct 11 12:11:08 2022] DELETE /identity/v3/users/a78e40d4eb554ca49eeec530a7adf428 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 12:11:08.482172 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1183/2357] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 12:11:08 2022] PUT /identity/v3/projects/6e42ea3c037b44b1a6dc2ca0a9ffdd6c/users/3434311c8240420896d6bfcb3428e515/roles/de2d2bd26fff4748a79990f97a107164 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:11:08.493577 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b229d82-7fed-4605-888b-b9378490109a None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:11:08.493861 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b229d82-7fed-4605-888b-b9378490109a None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:11:08.494353 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b229d82-7fed-4605-888b-b9378490109a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:11:08.524007 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-9b229d82-7fed-4605-888b-b9378490109a None None] MFA Rules not processed for user `3434311c8240420896d6bfcb3428e515`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:11:08.525186 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-9b229d82-7fed-4605-888b-b9378490109a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:11:08.603713 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1175/2358] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:11:08 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:11:08.614485 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb39e1c-f1ea-4ca6-8285-94c6e0dd47f5 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:11:08.616975 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-6cb39e1c-f1ea-4ca6-8285-94c6e0dd47f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:11:08.640281 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:11:08.662900 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b229d82-7fed-4605-888b-b9378490109a None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:11:08.664483 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-9b229d82-7fed-4605-888b-b9378490109a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:11:08.683580 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb39e1c-f1ea-4ca6-8285-94c6e0dd47f5 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:11:08.687316 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159d22d7-5bad-4f3f-b514-85b32862d1d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b1b651c73664d08accc6edf2f79bace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:11:08.687937 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159d22d7-5bad-4f3f-b514-85b32862d1d3 service cinder] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:11:08.688219 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159d22d7-5bad-4f3f-b514-85b32862d1d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:11:08.688512 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159d22d7-5bad-4f3f-b514-85b32862d1d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:11:08.697777 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159d22d7-5bad-4f3f-b514-85b32862d1d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:11:08.699361 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159d22d7-5bad-4f3f-b514-85b32862d1d3 service cinder] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:11:08.708374 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1184/2359] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:11:08 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:11:08.743947 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b229d82-7fed-4605-888b-b9378490109a None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:11:08.748802 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c7b189-571a-4c29-a13b-66e02862f8b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:11:08.749445 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c7b189-571a-4c29-a13b-66e02862f8b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:11:08.749737 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c7b189-571a-4c29-a13b-66e02862f8b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:11:08.749992 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c7b189-571a-4c29-a13b-66e02862f8b5 admin admin] PATH_INFO: `/v3/projects/9a7beb3a9d2b4ee9ba323d5bccc618c2` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:11:08.751170 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c7b189-571a-4c29-a13b-66e02862f8b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a7beb3a9d2b4ee9ba323d5bccc618c2)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:11:08.753073 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c7b189-571a-4c29-a13b-66e02862f8b5 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:11:08.804552 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-88c7b189-571a-4c29-a13b-66e02862f8b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a7beb3a9d2b4ee9ba323d5bccc618c2', 'request_id': 'req-88c7b189-571a-4c29-a13b-66e02862f8b5'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:11:08.811615 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-88c7b189-571a-4c29-a13b-66e02862f8b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a7beb3a9d2b4ee9ba323d5bccc618c2', 'request_id': 'req-88c7b189-571a-4c29-a13b-66e02862f8b5'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:11:08.820051 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-88c7b189-571a-4c29-a13b-66e02862f8b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a7beb3a9d2b4ee9ba323d5bccc618c2', 'request_id': 'req-88c7b189-571a-4c29-a13b-66e02862f8b5'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:11:08.833044 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-88c7b189-571a-4c29-a13b-66e02862f8b5 admin admin] The token cache is being invalidate because project 9a7beb3a9d2b4ee9ba323d5bccc618c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 12:11:08.833334 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-88c7b189-571a-4c29-a13b-66e02862f8b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:11:08.834016 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-88c7b189-571a-4c29-a13b-66e02862f8b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:11:08.835588 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1176/2360] 10.222.0.59 () {58 vars in 1233 bytes} [Tue Oct 11 12:11:08 2022] DELETE /identity/v3/projects/9a7beb3a9d2b4ee9ba323d5bccc618c2 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 12:11:08.840704 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4193f84-6e32-4905-805c-03830b76df1c None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:11:08.841002 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4193f84-6e32-4905-805c-03830b76df1c None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:11:08.841323 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4193f84-6e32-4905-805c-03830b76df1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:11:08.861868 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-f4193f84-6e32-4905-805c-03830b76df1c None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:11:08.863118 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-f4193f84-6e32-4905-805c-03830b76df1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:11:08.924811 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1185/2361] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:11:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 12:11:08.933120 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5cae43-e116-4f29-8a03-8da0414f3827 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:11:08.933416 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5cae43-e116-4f29-8a03-8da0414f3827 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:11:08.933681 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5cae43-e116-4f29-8a03-8da0414f3827 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:11:08.955250 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-5a5cae43-e116-4f29-8a03-8da0414f3827 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:11:08.956555 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-5a5cae43-e116-4f29-8a03-8da0414f3827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:11:08.965122 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1177/2362] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:11:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:11:08.969614 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4193f84-6e32-4905-805c-03830b76df1c None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:11:08.980233 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4193f84-6e32-4905-805c-03830b76df1c None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:11:08.985676 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e381dd53-f0e5-402e-a8de-d4e93996e776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:11:08.986306 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e381dd53-f0e5-402e-a8de-d4e93996e776 admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:11:08.986631 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e381dd53-f0e5-402e-a8de-d4e93996e776 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:11:08.986909 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e381dd53-f0e5-402e-a8de-d4e93996e776 admin admin] PATH_INFO: `/v3/domains` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:11:08.987566 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e381dd53-f0e5-402e-a8de-d4e93996e776 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 12:11:08.987868 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e381dd53-f0e5-402e-a8de-d4e93996e776 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:11:08.989497 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e381dd53-f0e5-402e-a8de-d4e93996e776 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:11:09.002795 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1186/2363] 10.222.0.59 () {58 vars in 1153 bytes} [Tue Oct 11 12:11:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 12:11:09.007093 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5cae43-e116-4f29-8a03-8da0414f3827 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:11:09.019740 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5cae43-e116-4f29-8a03-8da0414f3827 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:11:09.030649 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc2815c-8867-41f6-abfc-38c20b832a0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:11:09.030649 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc2815c-8867-41f6-abfc-38c20b832a0f admin admin] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:11:09.030649 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc2815c-8867-41f6-abfc-38c20b832a0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:11:09.030649 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc2815c-8867-41f6-abfc-38c20b832a0f admin admin] PATH_INFO: `/v3/projects` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:11:09.031705 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc2815c-8867-41f6-abfc-38c20b832a0f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:11:09.031705 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc2815c-8867-41f6-abfc-38c20b832a0f admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:11:09.036512 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:11:09.069931 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1178/2364] 10.222.0.59 () {60 vars in 1153 bytes} [Tue Oct 11 12:11:09 2022] POST /identity/v3/projects => generated 371 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 12:11:09.074401 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e381dd53-f0e5-402e-a8de-d4e93996e776 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:11:09.087246 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e381dd53-f0e5-402e-a8de-d4e93996e776 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:11:09.153227 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09379e88-005a-423f-884b-1d753763cbb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:11:09.153881 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09379e88-005a-423f-884b-1d753763cbb9 admin admin] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:11:09.154177 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09379e88-005a-423f-884b-1d753763cbb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:11:09.154443 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09379e88-005a-423f-884b-1d753763cbb9 admin admin] PATH_INFO: `/v3/users` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:11:09.155116 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09379e88-005a-423f-884b-1d753763cbb9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:11:09.157111 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09379e88-005a-423f-884b-1d753763cbb9 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:11:09.178638 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1187/2365] 10.222.0.59 () {60 vars in 1144 bytes} [Tue Oct 11 12:11:09 2022] POST /identity/v3/users => generated 286 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 12:11:09.182934 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc2815c-8867-41f6-abfc-38c20b832a0f admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:11:09.193788 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc2815c-8867-41f6-abfc-38c20b832a0f admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:11:09.199451 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503d8bfa-fd71-4e55-a8eb-a264734d881c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:11:09.200097 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503d8bfa-fd71-4e55-a8eb-a264734d881c admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:11:09.200380 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503d8bfa-fd71-4e55-a8eb-a264734d881c admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:11:09.200657 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503d8bfa-fd71-4e55-a8eb-a264734d881c admin admin] PATH_INFO: `/v3/roles` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:11:09.201351 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503d8bfa-fd71-4e55-a8eb-a264734d881c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:11:09.203093 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503d8bfa-fd71-4e55-a8eb-a264734d881c admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:11:09.214643 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1179/2366] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:11:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:11:09.218862 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09379e88-005a-423f-884b-1d753763cbb9 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:11:09.229251 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09379e88-005a-423f-884b-1d753763cbb9 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:11:09.234363 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced802e0-018a-4150-bd4f-9510f40840b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:11:09.235001 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced802e0-018a-4150-bd4f-9510f40840b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:11:09.235283 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced802e0-018a-4150-bd4f-9510f40840b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:11:09.235611 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced802e0-018a-4150-bd4f-9510f40840b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:11:09.236257 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced802e0-018a-4150-bd4f-9510f40840b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:11:09.237875 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced802e0-018a-4150-bd4f-9510f40840b3 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:11:09.249104 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1188/2367] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:11:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:11:09.255563 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503d8bfa-fd71-4e55-a8eb-a264734d881c admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:11:09.271485 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503d8bfa-fd71-4e55-a8eb-a264734d881c admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:11:09.278750 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f62d36-2505-434d-9323-1b9eeb02325e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:11:09.279408 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f62d36-2505-434d-9323-1b9eeb02325e admin admin] REQUEST_METHOD: `PUT` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:11:09.279748 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f62d36-2505-434d-9323-1b9eeb02325e admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:11:09.279972 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f62d36-2505-434d-9323-1b9eeb02325e admin admin] PATH_INFO: `/v3/projects/2f6be1ea606b4271b788ffb2aab7a780/users/acfa3e1cf291440ca346d5b78deb698e/roles/de2d2bd26fff4748a79990f97a107164` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:11:09.296165 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f62d36-2505-434d-9323-1b9eeb02325e admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f6be1ea606b4271b788ffb2aab7a780, user_id=acfa3e1cf291440ca346d5b78deb698e, role_id=de2d2bd26fff4748a79990f97a107164)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:11:09.298890 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f62d36-2505-434d-9323-1b9eeb02325e admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:11:09.320252 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1180/2368] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 12:11:09 2022] PUT /identity/v3/projects/2f6be1ea606b4271b788ffb2aab7a780/users/acfa3e1cf291440ca346d5b78deb698e/roles/de2d2bd26fff4748a79990f97a107164 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:11:09.335972 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74be1d6d-acde-4121-9661-d52633100a16 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:11:09.336505 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74be1d6d-acde-4121-9661-d52633100a16 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:11:09.336505 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74be1d6d-acde-4121-9661-d52633100a16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:11:09.371452 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-74be1d6d-acde-4121-9661-d52633100a16 None None] MFA Rules not processed for user `acfa3e1cf291440ca346d5b78deb698e`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:11:09.372684 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-74be1d6d-acde-4121-9661-d52633100a16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:11:09.390247 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:11:09.447456 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1189/2369] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:11:09 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:11:09.454505 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f62d36-2505-434d-9323-1b9eeb02325e admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:11:09.522017 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f62d36-2505-434d-9323-1b9eeb02325e admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:11:09.527124 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb48d4e-70c7-4ef7-9641-0712bf1a60a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:11:09.527730 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb48d4e-70c7-4ef7-9641-0712bf1a60a5 admin admin] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:11:09.528007 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb48d4e-70c7-4ef7-9641-0712bf1a60a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:11:09.528279 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb48d4e-70c7-4ef7-9641-0712bf1a60a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:11:09.528908 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb48d4e-70c7-4ef7-9641-0712bf1a60a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:11:09.530840 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb48d4e-70c7-4ef7-9641-0712bf1a60a5 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:11:09.561350 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1181/2370] 10.222.0.59 () {60 vars in 1153 bytes} [Tue Oct 11 12:11:09 2022] POST /identity/v3/projects => generated 373 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:11:09.565485 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74be1d6d-acde-4121-9661-d52633100a16 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:11:09.576471 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74be1d6d-acde-4121-9661-d52633100a16 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:11:09.635049 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55a6158-b0c1-47ca-b10c-c2d3afaca8a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:11:09.635655 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55a6158-b0c1-47ca-b10c-c2d3afaca8a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:11:09.635927 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55a6158-b0c1-47ca-b10c-c2d3afaca8a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:11:09.636175 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55a6158-b0c1-47ca-b10c-c2d3afaca8a8 admin admin] PATH_INFO: `/v3/users` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:11:09.636815 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55a6158-b0c1-47ca-b10c-c2d3afaca8a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:11:09.638843 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55a6158-b0c1-47ca-b10c-c2d3afaca8a8 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:11:09.666109 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1190/2371] 10.222.0.59 () {60 vars in 1144 bytes} [Tue Oct 11 12:11:09 2022] POST /identity/v3/users => generated 293 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:11:09.667379 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb48d4e-70c7-4ef7-9641-0712bf1a60a5 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:11:09.678481 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb48d4e-70c7-4ef7-9641-0712bf1a60a5 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:11:09.684101 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8d118e-e58c-43e5-8289-be8435b670f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:11:09.684680 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8d118e-e58c-43e5-8289-be8435b670f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:11:09.684954 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8d118e-e58c-43e5-8289-be8435b670f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:11:09.685251 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8d118e-e58c-43e5-8289-be8435b670f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:11:09.685881 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8d118e-e58c-43e5-8289-be8435b670f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:11:09.687534 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8d118e-e58c-43e5-8289-be8435b670f0 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:11:09.699456 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1182/2372] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:11:09 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 11 12:11:09.704518 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55a6158-b0c1-47ca-b10c-c2d3afaca8a8 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:11:09.718730 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55a6158-b0c1-47ca-b10c-c2d3afaca8a8 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:11:09.727809 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b129e204-9338-4726-962d-bf095d5ab209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:11:09.728860 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b129e204-9338-4726-962d-bf095d5ab209 admin admin] REQUEST_METHOD: `PUT` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:11:09.729250 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b129e204-9338-4726-962d-bf095d5ab209 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:11:09.729689 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b129e204-9338-4726-962d-bf095d5ab209 admin admin] PATH_INFO: `/v3/projects/9a366f0a65e040dfa01ae32fa7a48125/users/80d6dfe5fcd043d183fca7c6dc69eaf0/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:11:09.745739 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b129e204-9338-4726-962d-bf095d5ab209 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a366f0a65e040dfa01ae32fa7a48125, user_id=80d6dfe5fcd043d183fca7c6dc69eaf0, role_id=5cf1f4bd2fef46ab9c6eff56d52f9f69)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:11:09.748449 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b129e204-9338-4726-962d-bf095d5ab209 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:11:09.763910 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1191/2373] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 12:11:09 2022] PUT /identity/v3/projects/9a366f0a65e040dfa01ae32fa7a48125/users/80d6dfe5fcd043d183fca7c6dc69eaf0/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:11:09.768479 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8d118e-e58c-43e5-8289-be8435b670f0 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:11:09.779370 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8d118e-e58c-43e5-8289-be8435b670f0 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:11:09.844314 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a03a32b-1654-4232-ac9c-230e7abab549 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:11:09.844844 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a03a32b-1654-4232-ac9c-230e7abab549 admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:11:09.845128 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a03a32b-1654-4232-ac9c-230e7abab549 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:11:09.845410 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a03a32b-1654-4232-ac9c-230e7abab549 admin admin] PATH_INFO: `/v3/roles` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:11:09.846061 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a03a32b-1654-4232-ac9c-230e7abab549 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:11:09.847724 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a03a32b-1654-4232-ac9c-230e7abab549 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:11:09.858745 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1183/2374] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:11:09 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:11:09.863902 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b129e204-9338-4726-962d-bf095d5ab209 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:11:09.876903 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b129e204-9338-4726-962d-bf095d5ab209 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:11:09.882899 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2451f8fc-3299-4690-8b69-db8ecc1bb73b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:11:09.883561 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2451f8fc-3299-4690-8b69-db8ecc1bb73b admin admin] REQUEST_METHOD: `PUT` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:11:09.883846 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2451f8fc-3299-4690-8b69-db8ecc1bb73b admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:11:09.884113 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2451f8fc-3299-4690-8b69-db8ecc1bb73b admin admin] PATH_INFO: `/v3/projects/9a366f0a65e040dfa01ae32fa7a48125/users/80d6dfe5fcd043d183fca7c6dc69eaf0/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:11:09.886073 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2451f8fc-3299-4690-8b69-db8ecc1bb73b admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a366f0a65e040dfa01ae32fa7a48125, user_id=80d6dfe5fcd043d183fca7c6dc69eaf0, role_id=5cf1f4bd2fef46ab9c6eff56d52f9f69)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:11:09.888795 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2451f8fc-3299-4690-8b69-db8ecc1bb73b admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:11:09.899913 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1192/2375] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 12:11:09 2022] PUT /identity/v3/projects/9a366f0a65e040dfa01ae32fa7a48125/users/80d6dfe5fcd043d183fca7c6dc69eaf0/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:11:09.912760 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab1dad7-9ed1-4ba7-966d-043db73a2ccc None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:11:09.913290 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab1dad7-9ed1-4ba7-966d-043db73a2ccc None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:11:09.913746 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab1dad7-9ed1-4ba7-966d-043db73a2ccc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:11:09.946747 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-3ab1dad7-9ed1-4ba7-966d-043db73a2ccc None None] MFA Rules not processed for user `80d6dfe5fcd043d183fca7c6dc69eaf0`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:11:09.947965 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-3ab1dad7-9ed1-4ba7-966d-043db73a2ccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:11:09.956494 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:11:10.021620 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1184/2376] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:11:09 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:11:10.034304 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2451f8fc-3299-4690-8b69-db8ecc1bb73b admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:11:10.036446 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-2451f8fc-3299-4690-8b69-db8ecc1bb73b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:11:10.096843 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2451f8fc-3299-4690-8b69-db8ecc1bb73b admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:11:10.101754 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cdc9b9-c4be-48c8-a811-73b0a9d64341 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b1b651c73664d08accc6edf2f79bace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:11:10.102371 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cdc9b9-c4be-48c8-a811-73b0a9d64341 service cinder] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:11:10.102678 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cdc9b9-c4be-48c8-a811-73b0a9d64341 service cinder] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:11:10.102945 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cdc9b9-c4be-48c8-a811-73b0a9d64341 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:11:10.117420 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cdc9b9-c4be-48c8-a811-73b0a9d64341 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:11:10.117420 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cdc9b9-c4be-48c8-a811-73b0a9d64341 service cinder] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:11:10.129772 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1193/2377] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:11:10 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:11:10.550725 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab1dad7-9ed1-4ba7-966d-043db73a2ccc None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:11:10.615247 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab1dad7-9ed1-4ba7-966d-043db73a2ccc None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:11:10.620226 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2155fc86-32aa-4396-9cb4-6c0ef8220ae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:11:10.620862 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2155fc86-32aa-4396-9cb4-6c0ef8220ae4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:11:10.621138 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2155fc86-32aa-4396-9cb4-6c0ef8220ae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:11:10.621410 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2155fc86-32aa-4396-9cb4-6c0ef8220ae4 admin admin] PATH_INFO: `/v3/users/acfa3e1cf291440ca346d5b78deb698e` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:11:10.622701 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2155fc86-32aa-4396-9cb4-6c0ef8220ae4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=acfa3e1cf291440ca346d5b78deb698e)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:11:10.624761 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2155fc86-32aa-4396-9cb4-6c0ef8220ae4 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:11:10.674420 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-2155fc86-32aa-4396-9cb4-6c0ef8220ae4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'acfa3e1cf291440ca346d5b78deb698e'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:11:10.686602 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-2155fc86-32aa-4396-9cb4-6c0ef8220ae4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'acfa3e1cf291440ca346d5b78deb698e'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:11:10.695283 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-2155fc86-32aa-4396-9cb4-6c0ef8220ae4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'acfa3e1cf291440ca346d5b78deb698e'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:11:10.695884 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-2155fc86-32aa-4396-9cb4-6c0ef8220ae4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'acfa3e1cf291440ca346d5b78deb698e'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:11:10.703633 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-2155fc86-32aa-4396-9cb4-6c0ef8220ae4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'acfa3e1cf291440ca346d5b78deb698e'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:11:10.709526 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1185/2378] 10.222.0.59 () {58 vars in 1224 bytes} [Tue Oct 11 12:11:10 2022] DELETE /identity/v3/users/acfa3e1cf291440ca346d5b78deb698e => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:11:10.714405 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66687a24-b7a4-45f0-88c0-fe9b11233a9e None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:11:10.714946 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66687a24-b7a4-45f0-88c0-fe9b11233a9e None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:11:10.715044 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66687a24-b7a4-45f0-88c0-fe9b11233a9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:11:10.735239 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-66687a24-b7a4-45f0-88c0-fe9b11233a9e None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:11:10.736443 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-66687a24-b7a4-45f0-88c0-fe9b11233a9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:11:10.745650 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:11:10.808440 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1194/2379] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:11:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:11:10.819340 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2155fc86-32aa-4396-9cb4-6c0ef8220ae4 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:11:10.820938 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-2155fc86-32aa-4396-9cb4-6c0ef8220ae4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:11:10.851058 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:11:10.884661 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2155fc86-32aa-4396-9cb4-6c0ef8220ae4 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:11:10.888420 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03478a74-ac7b-437e-baaa-5772b0baede1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e563fe174db4cc19ae6bf8162cc3d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:11:10.889063 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03478a74-ac7b-437e-baaa-5772b0baede1 service neutron] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:11:10.889340 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03478a74-ac7b-437e-baaa-5772b0baede1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:11:10.889609 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03478a74-ac7b-437e-baaa-5772b0baede1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:11:10.899041 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03478a74-ac7b-437e-baaa-5772b0baede1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:11:10.900471 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03478a74-ac7b-437e-baaa-5772b0baede1 service neutron] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:11:10.909964 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1186/2380] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:11:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:11:11.194660 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66687a24-b7a4-45f0-88c0-fe9b11233a9e None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:11:11.196165 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-66687a24-b7a4-45f0-88c0-fe9b11233a9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:11:11.208908 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66687a24-b7a4-45f0-88c0-fe9b11233a9e None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:11:11.214178 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d8599d-1b62-47c0-9c14-276e9c6d1a82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:11:11.214889 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d8599d-1b62-47c0-9c14-276e9c6d1a82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:11:11.215173 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d8599d-1b62-47c0-9c14-276e9c6d1a82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:11:11.215450 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d8599d-1b62-47c0-9c14-276e9c6d1a82 admin admin] PATH_INFO: `/v3/projects/2f6be1ea606b4271b788ffb2aab7a780` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:11:11.216598 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d8599d-1b62-47c0-9c14-276e9c6d1a82 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f6be1ea606b4271b788ffb2aab7a780)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:11:11.218633 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d8599d-1b62-47c0-9c14-276e9c6d1a82 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:11:11.267129 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-a7d8599d-1b62-47c0-9c14-276e9c6d1a82 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f6be1ea606b4271b788ffb2aab7a780', 'request_id': 'req-a7d8599d-1b62-47c0-9c14-276e9c6d1a82'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:11:11.272954 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-a7d8599d-1b62-47c0-9c14-276e9c6d1a82 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f6be1ea606b4271b788ffb2aab7a780', 'request_id': 'req-a7d8599d-1b62-47c0-9c14-276e9c6d1a82'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:11:11.279899 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-a7d8599d-1b62-47c0-9c14-276e9c6d1a82 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f6be1ea606b4271b788ffb2aab7a780', 'request_id': 'req-a7d8599d-1b62-47c0-9c14-276e9c6d1a82'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:11:11.295275 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-a7d8599d-1b62-47c0-9c14-276e9c6d1a82 admin admin] The token cache is being invalidate because project 2f6be1ea606b4271b788ffb2aab7a780 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81432) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 12:11:11.295593 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-a7d8599d-1b62-47c0-9c14-276e9c6d1a82 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:11:11.296236 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-a7d8599d-1b62-47c0-9c14-276e9c6d1a82 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:11:11.297776 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1195/2381] 10.222.0.59 () {58 vars in 1233 bytes} [Tue Oct 11 12:11:11 2022] DELETE /identity/v3/projects/2f6be1ea606b4271b788ffb2aab7a780 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:11:11.301903 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03478a74-ac7b-437e-baaa-5772b0baede1 service neutron] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:11:11.313671 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03478a74-ac7b-437e-baaa-5772b0baede1 service neutron] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:11:11.379720 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735d4555-3107-40f4-83be-edc723112b72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:11:11.380358 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735d4555-3107-40f4-83be-edc723112b72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:11:11.380625 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735d4555-3107-40f4-83be-edc723112b72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:11:11.380914 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735d4555-3107-40f4-83be-edc723112b72 admin admin] PATH_INFO: `/v3/users/80d6dfe5fcd043d183fca7c6dc69eaf0` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:11:11.382136 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735d4555-3107-40f4-83be-edc723112b72 admin admin] RBAC: Authorizing `identity:delete_user(user_id=80d6dfe5fcd043d183fca7c6dc69eaf0)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:11:11.384122 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735d4555-3107-40f4-83be-edc723112b72 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:11:11.435489 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-735d4555-3107-40f4-83be-edc723112b72 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80d6dfe5fcd043d183fca7c6dc69eaf0'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:11:11.447840 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-735d4555-3107-40f4-83be-edc723112b72 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80d6dfe5fcd043d183fca7c6dc69eaf0'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:11:11.458052 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-735d4555-3107-40f4-83be-edc723112b72 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80d6dfe5fcd043d183fca7c6dc69eaf0'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:11:11.458862 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-735d4555-3107-40f4-83be-edc723112b72 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80d6dfe5fcd043d183fca7c6dc69eaf0'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:11:11.468919 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-735d4555-3107-40f4-83be-edc723112b72 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80d6dfe5fcd043d183fca7c6dc69eaf0'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:11:11.475654 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1187/2382] 10.222.0.59 () {58 vars in 1224 bytes} [Tue Oct 11 12:11:11 2022] DELETE /identity/v3/users/80d6dfe5fcd043d183fca7c6dc69eaf0 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 12:11:11.777573 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d8599d-1b62-47c0-9c14-276e9c6d1a82 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:11:11.779292 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-a7d8599d-1b62-47c0-9c14-276e9c6d1a82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:11:11.796431 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d8599d-1b62-47c0-9c14-276e9c6d1a82 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:11:11.856540 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0cd6d1-b4c4-4aa9-8232-a47420e14975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:11:11.857238 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0cd6d1-b4c4-4aa9-8232-a47420e14975 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:11:11.857526 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0cd6d1-b4c4-4aa9-8232-a47420e14975 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:11:11.857800 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0cd6d1-b4c4-4aa9-8232-a47420e14975 admin admin] PATH_INFO: `/v3/projects/9a366f0a65e040dfa01ae32fa7a48125` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:11:11.859044 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0cd6d1-b4c4-4aa9-8232-a47420e14975 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a366f0a65e040dfa01ae32fa7a48125)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:11:11.860987 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0cd6d1-b4c4-4aa9-8232-a47420e14975 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:11:11.904898 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:11:11.913608 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-cf0cd6d1-b4c4-4aa9-8232-a47420e14975 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a366f0a65e040dfa01ae32fa7a48125', 'request_id': 'req-cf0cd6d1-b4c4-4aa9-8232-a47420e14975'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:11:11.920797 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-cf0cd6d1-b4c4-4aa9-8232-a47420e14975 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a366f0a65e040dfa01ae32fa7a48125', 'request_id': 'req-cf0cd6d1-b4c4-4aa9-8232-a47420e14975'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:11:11.928286 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-cf0cd6d1-b4c4-4aa9-8232-a47420e14975 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a366f0a65e040dfa01ae32fa7a48125', 'request_id': 'req-cf0cd6d1-b4c4-4aa9-8232-a47420e14975'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:11:11.951395 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-cf0cd6d1-b4c4-4aa9-8232-a47420e14975 admin admin] The token cache is being invalidate because project 9a366f0a65e040dfa01ae32fa7a48125 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81432) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 12:11:11.951652 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-cf0cd6d1-b4c4-4aa9-8232-a47420e14975 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:11:11.952452 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-cf0cd6d1-b4c4-4aa9-8232-a47420e14975 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:11:11.953934 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1196/2383] 10.222.0.59 () {58 vars in 1233 bytes} [Tue Oct 11 12:11:11 2022] DELETE /identity/v3/projects/9a366f0a65e040dfa01ae32fa7a48125 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:11:11.958993 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8e3a2a-2c1d-4fc5-9547-988345e82cab None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:11:11.959512 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8e3a2a-2c1d-4fc5-9547-988345e82cab None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:11:11.959512 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8e3a2a-2c1d-4fc5-9547-988345e82cab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:11:11.981375 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-7f8e3a2a-2c1d-4fc5-9547-988345e82cab None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:11:11.982630 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-7f8e3a2a-2c1d-4fc5-9547-988345e82cab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:11:12.025703 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:11:12.045835 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1188/2384] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:11:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:11:12.053910 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8285ab-f530-48d1-b56a-50363db0c0c7 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:11:12.054285 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8285ab-f530-48d1-b56a-50363db0c0c7 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:11:12.054640 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8285ab-f530-48d1-b56a-50363db0c0c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:11:12.076909 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-de8285ab-f530-48d1-b56a-50363db0c0c7 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:11:12.078142 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-de8285ab-f530-48d1-b56a-50363db0c0c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:11:12.087703 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1197/2385] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:11:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:11:12.092173 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8e3a2a-2c1d-4fc5-9547-988345e82cab None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:11:12.102103 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8e3a2a-2c1d-4fc5-9547-988345e82cab None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:11:12.107086 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f20fcb1-d6f3-4720-a6a1-9f9f9652243b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:11:12.107682 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f20fcb1-d6f3-4720-a6a1-9f9f9652243b admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:11:12.107961 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f20fcb1-d6f3-4720-a6a1-9f9f9652243b admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:11:12.108230 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f20fcb1-d6f3-4720-a6a1-9f9f9652243b admin admin] PATH_INFO: `/v3/domains` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:11:12.108928 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f20fcb1-d6f3-4720-a6a1-9f9f9652243b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 12:11:12.109855 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f20fcb1-d6f3-4720-a6a1-9f9f9652243b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:11:12.111590 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f20fcb1-d6f3-4720-a6a1-9f9f9652243b admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:11:12.123486 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1189/2386] 10.222.0.59 () {58 vars in 1153 bytes} [Tue Oct 11 12:11:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 12:11:12.127794 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8285ab-f530-48d1-b56a-50363db0c0c7 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:11:12.137184 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8285ab-f530-48d1-b56a-50363db0c0c7 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:11:12.143329 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4605f104-92c6-41c1-be5c-2980981ba3f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:11:12.143936 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4605f104-92c6-41c1-be5c-2980981ba3f5 admin admin] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:11:12.144222 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4605f104-92c6-41c1-be5c-2980981ba3f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:11:12.144486 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4605f104-92c6-41c1-be5c-2980981ba3f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:11:12.145117 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4605f104-92c6-41c1-be5c-2980981ba3f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:11:12.147093 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4605f104-92c6-41c1-be5c-2980981ba3f5 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:11:12.178260 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1198/2387] 10.222.0.59 () {60 vars in 1153 bytes} [Tue Oct 11 12:11:12 2022] POST /identity/v3/projects => generated 381 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:11:12.182717 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f20fcb1-d6f3-4720-a6a1-9f9f9652243b admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:11:12.193164 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f20fcb1-d6f3-4720-a6a1-9f9f9652243b admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:11:12.249858 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd6fffd-6468-4625-833f-1c4540598ae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:11:12.250389 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd6fffd-6468-4625-833f-1c4540598ae4 admin admin] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:11:12.250701 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd6fffd-6468-4625-833f-1c4540598ae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:11:12.250976 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd6fffd-6468-4625-833f-1c4540598ae4 admin admin] PATH_INFO: `/v3/users` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:11:12.251601 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd6fffd-6468-4625-833f-1c4540598ae4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:11:12.253526 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd6fffd-6468-4625-833f-1c4540598ae4 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:11:12.281938 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1190/2388] 10.222.0.59 () {60 vars in 1144 bytes} [Tue Oct 11 12:11:12 2022] POST /identity/v3/users => generated 291 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:11:12.286223 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4605f104-92c6-41c1-be5c-2980981ba3f5 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:11:12.297169 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4605f104-92c6-41c1-be5c-2980981ba3f5 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:11:12.302906 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66292669-e26a-4392-9644-5a083f49bddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:11:12.303688 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66292669-e26a-4392-9644-5a083f49bddf admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:11:12.304144 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66292669-e26a-4392-9644-5a083f49bddf admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:11:12.304520 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66292669-e26a-4392-9644-5a083f49bddf admin admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:11:12.305353 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66292669-e26a-4392-9644-5a083f49bddf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:11:12.307089 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66292669-e26a-4392-9644-5a083f49bddf admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:11:12.317493 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1199/2389] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:11:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:11:12.322035 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd6fffd-6468-4625-833f-1c4540598ae4 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:11:12.332539 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd6fffd-6468-4625-833f-1c4540598ae4 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:11:12.337995 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74afb14-0510-4779-bcc8-117ef0b99f25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:11:12.338627 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74afb14-0510-4779-bcc8-117ef0b99f25 admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:11:12.338908 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74afb14-0510-4779-bcc8-117ef0b99f25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:11:12.339172 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74afb14-0510-4779-bcc8-117ef0b99f25 admin admin] PATH_INFO: `/v3/roles` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:11:12.339794 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74afb14-0510-4779-bcc8-117ef0b99f25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:11:12.341407 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74afb14-0510-4779-bcc8-117ef0b99f25 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:11:12.352327 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1191/2390] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:11:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:11:12.356729 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66292669-e26a-4392-9644-5a083f49bddf admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:11:12.367286 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66292669-e26a-4392-9644-5a083f49bddf admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:11:12.372963 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3639c9ee-3df1-4f5f-a466-882b74ab9a0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:11:12.373662 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3639c9ee-3df1-4f5f-a466-882b74ab9a0a admin admin] REQUEST_METHOD: `PUT` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:11:12.373941 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3639c9ee-3df1-4f5f-a466-882b74ab9a0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:11:12.374421 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3639c9ee-3df1-4f5f-a466-882b74ab9a0a admin admin] PATH_INFO: `/v3/projects/51c001d288d44b0cace0dd13a2e8e68e/users/8ad5be730f4c413aa5c089ba73f75d90/roles/de2d2bd26fff4748a79990f97a107164` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:11:12.392104 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3639c9ee-3df1-4f5f-a466-882b74ab9a0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=51c001d288d44b0cace0dd13a2e8e68e, user_id=8ad5be730f4c413aa5c089ba73f75d90, role_id=de2d2bd26fff4748a79990f97a107164)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:11:12.394872 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3639c9ee-3df1-4f5f-a466-882b74ab9a0a admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:11:12.410134 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1200/2391] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 12:11:12 2022] PUT /identity/v3/projects/51c001d288d44b0cace0dd13a2e8e68e/users/8ad5be730f4c413aa5c089ba73f75d90/roles/de2d2bd26fff4748a79990f97a107164 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:11:12.422160 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8c9a56-1a04-4022-88f8-fb033e825973 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:11:12.422454 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8c9a56-1a04-4022-88f8-fb033e825973 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:11:12.422757 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8c9a56-1a04-4022-88f8-fb033e825973 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:11:12.452826 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-8f8c9a56-1a04-4022-88f8-fb033e825973 None None] MFA Rules not processed for user `8ad5be730f4c413aa5c089ba73f75d90`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:11:12.454058 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-8f8c9a56-1a04-4022-88f8-fb033e825973 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:11:12.534043 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1192/2392] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:11:12 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:11:12.540872 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3639c9ee-3df1-4f5f-a466-882b74ab9a0a admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:11:12.608200 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3639c9ee-3df1-4f5f-a466-882b74ab9a0a admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:11:12.614103 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c64b9b0-d8d1-4e08-8ad9-965bb1046bd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:11:12.614744 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c64b9b0-d8d1-4e08-8ad9-965bb1046bd5 admin admin] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:11:12.615024 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c64b9b0-d8d1-4e08-8ad9-965bb1046bd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:11:12.615264 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c64b9b0-d8d1-4e08-8ad9-965bb1046bd5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:11:12.615894 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c64b9b0-d8d1-4e08-8ad9-965bb1046bd5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:11:12.618017 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c64b9b0-d8d1-4e08-8ad9-965bb1046bd5 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:11:12.648889 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1201/2393] 10.222.0.59 () {60 vars in 1153 bytes} [Tue Oct 11 12:11:12 2022] POST /identity/v3/projects => generated 379 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:11:12.653263 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8c9a56-1a04-4022-88f8-fb033e825973 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:11:12.665422 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8c9a56-1a04-4022-88f8-fb033e825973 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:11:12.708671 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:11:12.727156 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406a7476-9153-4cec-981d-594fd5b5276f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:11:12.727789 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406a7476-9153-4cec-981d-594fd5b5276f admin admin] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:11:12.728070 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406a7476-9153-4cec-981d-594fd5b5276f admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:11:12.728340 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406a7476-9153-4cec-981d-594fd5b5276f admin admin] PATH_INFO: `/v3/users` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:11:12.728974 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406a7476-9153-4cec-981d-594fd5b5276f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:11:12.730939 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406a7476-9153-4cec-981d-594fd5b5276f admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:11:12.753984 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1193/2394] 10.222.0.59 () {60 vars in 1144 bytes} [Tue Oct 11 12:11:12 2022] POST /identity/v3/users => generated 296 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:11:12.758115 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c64b9b0-d8d1-4e08-8ad9-965bb1046bd5 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:11:12.769665 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c64b9b0-d8d1-4e08-8ad9-965bb1046bd5 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:11:12.775382 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963e335b-8ebe-4a61-a1d4-5d15e9b0f605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:11:12.775987 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963e335b-8ebe-4a61-a1d4-5d15e9b0f605 admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:11:12.776260 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963e335b-8ebe-4a61-a1d4-5d15e9b0f605 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:11:12.776554 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963e335b-8ebe-4a61-a1d4-5d15e9b0f605 admin admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:11:12.777202 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963e335b-8ebe-4a61-a1d4-5d15e9b0f605 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:11:12.778867 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963e335b-8ebe-4a61-a1d4-5d15e9b0f605 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:11:12.790205 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1202/2395] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:11:12 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:11:12.794897 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406a7476-9153-4cec-981d-594fd5b5276f admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:11:12.805597 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406a7476-9153-4cec-981d-594fd5b5276f admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:11:12.810998 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8cb325-b6c8-44e7-82d8-6aefdf6fdc70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:11:12.811657 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8cb325-b6c8-44e7-82d8-6aefdf6fdc70 admin admin] REQUEST_METHOD: `PUT` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:11:12.811981 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8cb325-b6c8-44e7-82d8-6aefdf6fdc70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:11:12.812260 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8cb325-b6c8-44e7-82d8-6aefdf6fdc70 admin admin] PATH_INFO: `/v3/projects/fd7020a079194c54ad285204e37d6a9a/users/770922ca2e0f4394b5dfdf296de4a82f/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:11:12.827749 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8cb325-b6c8-44e7-82d8-6aefdf6fdc70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd7020a079194c54ad285204e37d6a9a, user_id=770922ca2e0f4394b5dfdf296de4a82f, role_id=5cf1f4bd2fef46ab9c6eff56d52f9f69)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:11:12.830459 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8cb325-b6c8-44e7-82d8-6aefdf6fdc70 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:11:12.846733 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1194/2396] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 12:11:12 2022] PUT /identity/v3/projects/fd7020a079194c54ad285204e37d6a9a/users/770922ca2e0f4394b5dfdf296de4a82f/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:11:12.851012 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963e335b-8ebe-4a61-a1d4-5d15e9b0f605 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:11:12.861474 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963e335b-8ebe-4a61-a1d4-5d15e9b0f605 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:11:12.899570 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:11:12.924720 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c161bd5-6c45-4c33-810f-4ee5c632e97b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:11:12.925346 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c161bd5-6c45-4c33-810f-4ee5c632e97b admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:11:12.925624 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c161bd5-6c45-4c33-810f-4ee5c632e97b admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:11:12.925889 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c161bd5-6c45-4c33-810f-4ee5c632e97b admin admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:11:12.926523 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c161bd5-6c45-4c33-810f-4ee5c632e97b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:11:12.928197 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c161bd5-6c45-4c33-810f-4ee5c632e97b admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:11:12.939004 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1203/2397] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:11:12 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:11:12.943249 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8cb325-b6c8-44e7-82d8-6aefdf6fdc70 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:11:12.954191 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8cb325-b6c8-44e7-82d8-6aefdf6fdc70 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:11:12.963807 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e454ce9-b1c6-432a-bbb3-db137bc6c150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:11:12.964547 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e454ce9-b1c6-432a-bbb3-db137bc6c150 admin admin] REQUEST_METHOD: `PUT` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:11:12.964745 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e454ce9-b1c6-432a-bbb3-db137bc6c150 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:11:12.965313 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e454ce9-b1c6-432a-bbb3-db137bc6c150 admin admin] PATH_INFO: `/v3/projects/fd7020a079194c54ad285204e37d6a9a/users/770922ca2e0f4394b5dfdf296de4a82f/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:11:12.967076 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e454ce9-b1c6-432a-bbb3-db137bc6c150 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd7020a079194c54ad285204e37d6a9a, user_id=770922ca2e0f4394b5dfdf296de4a82f, role_id=5cf1f4bd2fef46ab9c6eff56d52f9f69)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:11:12.969810 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e454ce9-b1c6-432a-bbb3-db137bc6c150 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:11:12.981279 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1195/2398] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 12:11:12 2022] PUT /identity/v3/projects/fd7020a079194c54ad285204e37d6a9a/users/770922ca2e0f4394b5dfdf296de4a82f/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 12:11:12.991900 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69738811-ff8c-4f2f-b656-4b59a09831ae None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:11:12.992270 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69738811-ff8c-4f2f-b656-4b59a09831ae None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:11:12.992608 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69738811-ff8c-4f2f-b656-4b59a09831ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:11:13.025525 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-69738811-ff8c-4f2f-b656-4b59a09831ae None None] MFA Rules not processed for user `770922ca2e0f4394b5dfdf296de4a82f`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:11:13.026806 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-69738811-ff8c-4f2f-b656-4b59a09831ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:11:13.100811 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1204/2399] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:11:12 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:11:13.112553 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e454ce9-b1c6-432a-bbb3-db137bc6c150 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:11:13.114586 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-6e454ce9-b1c6-432a-bbb3-db137bc6c150 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:11:13.176554 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e454ce9-b1c6-432a-bbb3-db137bc6c150 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:11:13.180510 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c493fe31-8e4a-4147-89fe-964cfe22ef49 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b1b651c73664d08accc6edf2f79bace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:11:13.181118 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c493fe31-8e4a-4147-89fe-964cfe22ef49 service cinder] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:11:13.181398 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c493fe31-8e4a-4147-89fe-964cfe22ef49 service cinder] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:11:13.181670 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c493fe31-8e4a-4147-89fe-964cfe22ef49 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:11:13.191310 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c493fe31-8e4a-4147-89fe-964cfe22ef49 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:11:13.192753 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c493fe31-8e4a-4147-89fe-964cfe22ef49 service cinder] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:11:13.202595 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1196/2400] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:11:13 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 11 12:11:13.615074 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69738811-ff8c-4f2f-b656-4b59a09831ae None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:11:13.682521 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69738811-ff8c-4f2f-b656-4b59a09831ae None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:11:13.687041 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2dc846-72dc-4a64-9acb-079d67b3b906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:11:13.687683 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2dc846-72dc-4a64-9acb-079d67b3b906 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:11:13.687961 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2dc846-72dc-4a64-9acb-079d67b3b906 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:11:13.688235 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2dc846-72dc-4a64-9acb-079d67b3b906 admin admin] PATH_INFO: `/v3/users/8ad5be730f4c413aa5c089ba73f75d90` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:11:13.689360 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2dc846-72dc-4a64-9acb-079d67b3b906 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ad5be730f4c413aa5c089ba73f75d90)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:11:13.691339 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2dc846-72dc-4a64-9acb-079d67b3b906 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:11:13.740867 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-dc2dc846-72dc-4a64-9acb-079d67b3b906 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ad5be730f4c413aa5c089ba73f75d90'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:11:13.751769 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:11:13.754344 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-dc2dc846-72dc-4a64-9acb-079d67b3b906 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ad5be730f4c413aa5c089ba73f75d90'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:11:13.755538 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-dc2dc846-72dc-4a64-9acb-079d67b3b906 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ad5be730f4c413aa5c089ba73f75d90'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:11:13.764857 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-dc2dc846-72dc-4a64-9acb-079d67b3b906 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ad5be730f4c413aa5c089ba73f75d90'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:11:13.765679 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-dc2dc846-72dc-4a64-9acb-079d67b3b906 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ad5be730f4c413aa5c089ba73f75d90'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:11:13.779188 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1205/2401] 10.222.0.59 () {58 vars in 1224 bytes} [Tue Oct 11 12:11:13 2022] DELETE /identity/v3/users/8ad5be730f4c413aa5c089ba73f75d90 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:11:13.783951 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf66889b-4e74-4aa9-9d0a-67eb9c7bbb18 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:11:13.784323 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf66889b-4e74-4aa9-9d0a-67eb9c7bbb18 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:11:13.784662 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf66889b-4e74-4aa9-9d0a-67eb9c7bbb18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:11:13.805296 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-cf66889b-4e74-4aa9-9d0a-67eb9c7bbb18 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:11:13.806527 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-cf66889b-4e74-4aa9-9d0a-67eb9c7bbb18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:11:13.878865 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1197/2402] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:11:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:11:13.886235 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2dc846-72dc-4a64-9acb-079d67b3b906 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:11:13.888038 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-dc2dc846-72dc-4a64-9acb-079d67b3b906 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:11:13.950713 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2dc846-72dc-4a64-9acb-079d67b3b906 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:11:13.955289 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc9d4aa-6685-43f8-bce0-1f52f03e7de9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e563fe174db4cc19ae6bf8162cc3d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:11:13.955924 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc9d4aa-6685-43f8-bce0-1f52f03e7de9 service neutron] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:11:13.956200 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc9d4aa-6685-43f8-bce0-1f52f03e7de9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:11:13.957214 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc9d4aa-6685-43f8-bce0-1f52f03e7de9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:11:13.967588 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc9d4aa-6685-43f8-bce0-1f52f03e7de9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:11:13.969056 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc9d4aa-6685-43f8-bce0-1f52f03e7de9 service neutron] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:11:13.979481 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1206/2403] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:11:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:11:14.257512 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf66889b-4e74-4aa9-9d0a-67eb9c7bbb18 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:11:14.259252 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-cf66889b-4e74-4aa9-9d0a-67eb9c7bbb18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:11:14.278662 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf66889b-4e74-4aa9-9d0a-67eb9c7bbb18 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:11:14.284029 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c62985-6ce0-4a6a-8351-0931272de983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:11:14.284691 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c62985-6ce0-4a6a-8351-0931272de983 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:11:14.284971 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c62985-6ce0-4a6a-8351-0931272de983 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:11:14.285728 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c62985-6ce0-4a6a-8351-0931272de983 admin admin] PATH_INFO: `/v3/projects/51c001d288d44b0cace0dd13a2e8e68e` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:11:14.286479 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c62985-6ce0-4a6a-8351-0931272de983 admin admin] RBAC: Authorizing `identity:delete_project(project_id=51c001d288d44b0cace0dd13a2e8e68e)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:11:14.288433 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c62985-6ce0-4a6a-8351-0931272de983 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:11:14.333854 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:11:14.348619 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-91c62985-6ce0-4a6a-8351-0931272de983 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '51c001d288d44b0cace0dd13a2e8e68e', 'request_id': 'req-91c62985-6ce0-4a6a-8351-0931272de983'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:11:14.356814 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-91c62985-6ce0-4a6a-8351-0931272de983 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '51c001d288d44b0cace0dd13a2e8e68e', 'request_id': 'req-91c62985-6ce0-4a6a-8351-0931272de983'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:11:14.365680 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-91c62985-6ce0-4a6a-8351-0931272de983 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '51c001d288d44b0cace0dd13a2e8e68e', 'request_id': 'req-91c62985-6ce0-4a6a-8351-0931272de983'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:11:14.377540 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-91c62985-6ce0-4a6a-8351-0931272de983 admin admin] The token cache is being invalidate because project 51c001d288d44b0cace0dd13a2e8e68e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 12:11:14.377852 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-91c62985-6ce0-4a6a-8351-0931272de983 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:11:14.378459 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-91c62985-6ce0-4a6a-8351-0931272de983 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:11:14.379984 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1198/2404] 10.222.0.59 () {58 vars in 1233 bytes} [Tue Oct 11 12:11:14 2022] DELETE /identity/v3/projects/51c001d288d44b0cace0dd13a2e8e68e => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:11:14.388206 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc9d4aa-6685-43f8-bce0-1f52f03e7de9 service neutron] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:11:14.399660 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc9d4aa-6685-43f8-bce0-1f52f03e7de9 service neutron] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:11:14.466254 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6665f204-0570-44ab-87f7-ae16c8b2bb9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:11:14.466950 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6665f204-0570-44ab-87f7-ae16c8b2bb9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:11:14.467206 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6665f204-0570-44ab-87f7-ae16c8b2bb9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:11:14.467470 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6665f204-0570-44ab-87f7-ae16c8b2bb9c admin admin] PATH_INFO: `/v3/users/770922ca2e0f4394b5dfdf296de4a82f` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:11:14.468689 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6665f204-0570-44ab-87f7-ae16c8b2bb9c admin admin] RBAC: Authorizing `identity:delete_user(user_id=770922ca2e0f4394b5dfdf296de4a82f)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:11:14.470644 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6665f204-0570-44ab-87f7-ae16c8b2bb9c admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:11:14.520359 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-6665f204-0570-44ab-87f7-ae16c8b2bb9c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '770922ca2e0f4394b5dfdf296de4a82f'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:11:14.532496 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-6665f204-0570-44ab-87f7-ae16c8b2bb9c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '770922ca2e0f4394b5dfdf296de4a82f'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:11:14.533249 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-6665f204-0570-44ab-87f7-ae16c8b2bb9c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '770922ca2e0f4394b5dfdf296de4a82f'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:11:14.541535 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-6665f204-0570-44ab-87f7-ae16c8b2bb9c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '770922ca2e0f4394b5dfdf296de4a82f'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:11:14.542240 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-6665f204-0570-44ab-87f7-ae16c8b2bb9c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '770922ca2e0f4394b5dfdf296de4a82f'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:11:14.551098 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:11:14.557950 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1207/2405] 10.222.0.59 () {58 vars in 1224 bytes} [Tue Oct 11 12:11:14 2022] DELETE /identity/v3/users/770922ca2e0f4394b5dfdf296de4a82f => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:11:14.838638 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c62985-6ce0-4a6a-8351-0931272de983 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:11:14.840262 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-91c62985-6ce0-4a6a-8351-0931272de983 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:11:14.857081 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c62985-6ce0-4a6a-8351-0931272de983 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:11:14.920451 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fade8e5-0897-42ad-b21f-26a88defff20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:11:14.921477 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fade8e5-0897-42ad-b21f-26a88defff20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:11:14.921765 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fade8e5-0897-42ad-b21f-26a88defff20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:11:14.922069 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fade8e5-0897-42ad-b21f-26a88defff20 admin admin] PATH_INFO: `/v3/projects/fd7020a079194c54ad285204e37d6a9a` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:11:14.923273 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fade8e5-0897-42ad-b21f-26a88defff20 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd7020a079194c54ad285204e37d6a9a)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:11:14.925259 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fade8e5-0897-42ad-b21f-26a88defff20 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:11:14.980899 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-8fade8e5-0897-42ad-b21f-26a88defff20 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd7020a079194c54ad285204e37d6a9a', 'request_id': 'req-8fade8e5-0897-42ad-b21f-26a88defff20'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:11:14.986756 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-8fade8e5-0897-42ad-b21f-26a88defff20 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd7020a079194c54ad285204e37d6a9a', 'request_id': 'req-8fade8e5-0897-42ad-b21f-26a88defff20'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:11:14.995627 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-8fade8e5-0897-42ad-b21f-26a88defff20 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd7020a079194c54ad285204e37d6a9a', 'request_id': 'req-8fade8e5-0897-42ad-b21f-26a88defff20'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:11:15.007708 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-8fade8e5-0897-42ad-b21f-26a88defff20 admin admin] The token cache is being invalidate because project fd7020a079194c54ad285204e37d6a9a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 12:11:15.007997 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-8fade8e5-0897-42ad-b21f-26a88defff20 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:11:15.008631 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-8fade8e5-0897-42ad-b21f-26a88defff20 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:11:15.010384 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1199/2406] 10.222.0.59 () {58 vars in 1233 bytes} [Tue Oct 11 12:11:14 2022] DELETE /identity/v3/projects/fd7020a079194c54ad285204e37d6a9a => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:11:15.015355 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a838bc6c-24fb-4b00-89f4-687b6cab57ea None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:11:15.015843 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a838bc6c-24fb-4b00-89f4-687b6cab57ea None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:11:15.015958 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a838bc6c-24fb-4b00-89f4-687b6cab57ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:11:15.036975 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-a838bc6c-24fb-4b00-89f4-687b6cab57ea None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:11:15.038218 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-a838bc6c-24fb-4b00-89f4-687b6cab57ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:11:15.101326 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1208/2407] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:11:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:11:15.109387 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf059496-1a1a-4cf3-9b2c-32541b6868ff None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:11:15.109739 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf059496-1a1a-4cf3-9b2c-32541b6868ff None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:11:15.109973 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf059496-1a1a-4cf3-9b2c-32541b6868ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:11:15.130506 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-bf059496-1a1a-4cf3-9b2c-32541b6868ff None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:11:15.131923 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-bf059496-1a1a-4cf3-9b2c-32541b6868ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:11:15.140651 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1200/2408] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:11:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:11:15.145475 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a838bc6c-24fb-4b00-89f4-687b6cab57ea None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:11:15.155982 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a838bc6c-24fb-4b00-89f4-687b6cab57ea None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:11:15.160931 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfeea86a-2b39-49b5-add4-8b067224c507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:11:15.161545 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeea86a-2b39-49b5-add4-8b067224c507 admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:11:15.161820 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeea86a-2b39-49b5-add4-8b067224c507 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:11:15.162093 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeea86a-2b39-49b5-add4-8b067224c507 admin admin] PATH_INFO: `/v3/domains` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:11:15.162762 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfeea86a-2b39-49b5-add4-8b067224c507 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 12:11:15.163045 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfeea86a-2b39-49b5-add4-8b067224c507 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:11:15.164643 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfeea86a-2b39-49b5-add4-8b067224c507 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:11:15.177586 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1209/2409] 10.222.0.59 () {58 vars in 1153 bytes} [Tue Oct 11 12:11:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 12:11:15.182008 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf059496-1a1a-4cf3-9b2c-32541b6868ff None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:11:15.191801 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf059496-1a1a-4cf3-9b2c-32541b6868ff None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:11:15.196620 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ff15a0-4970-4ae8-bc36-5cec73c3f487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:11:15.197228 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ff15a0-4970-4ae8-bc36-5cec73c3f487 admin admin] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:11:15.197507 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ff15a0-4970-4ae8-bc36-5cec73c3f487 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:11:15.197778 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ff15a0-4970-4ae8-bc36-5cec73c3f487 admin admin] PATH_INFO: `/v3/projects` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:11:15.198403 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ff15a0-4970-4ae8-bc36-5cec73c3f487 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:11:15.200326 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ff15a0-4970-4ae8-bc36-5cec73c3f487 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:11:15.229343 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1201/2410] 10.222.0.59 () {60 vars in 1153 bytes} [Tue Oct 11 12:11:15 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:11:15.234286 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfeea86a-2b39-49b5-add4-8b067224c507 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:11:15.244871 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfeea86a-2b39-49b5-add4-8b067224c507 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:11:15.312483 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6065f4e6-4515-44f9-ba90-bdcc0b8e8a32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:11:15.313102 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6065f4e6-4515-44f9-ba90-bdcc0b8e8a32 admin admin] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:11:15.313383 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6065f4e6-4515-44f9-ba90-bdcc0b8e8a32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:11:15.314916 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6065f4e6-4515-44f9-ba90-bdcc0b8e8a32 admin admin] PATH_INFO: `/v3/users` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:11:15.315596 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6065f4e6-4515-44f9-ba90-bdcc0b8e8a32 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:11:15.317583 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6065f4e6-4515-44f9-ba90-bdcc0b8e8a32 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:11:15.340208 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1210/2411] 10.222.0.59 () {60 vars in 1144 bytes} [Tue Oct 11 12:11:15 2022] POST /identity/v3/users => generated 288 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:11:15.344638 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ff15a0-4970-4ae8-bc36-5cec73c3f487 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:11:15.355527 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ff15a0-4970-4ae8-bc36-5cec73c3f487 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:11:15.360262 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ca3239-5cc8-4adf-8326-0e4ec2913235 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:11:15.360927 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ca3239-5cc8-4adf-8326-0e4ec2913235 admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:11:15.361520 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ca3239-5cc8-4adf-8326-0e4ec2913235 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:11:15.361864 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ca3239-5cc8-4adf-8326-0e4ec2913235 admin admin] PATH_INFO: `/v3/roles` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:11:15.362578 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ca3239-5cc8-4adf-8326-0e4ec2913235 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:11:15.364278 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ca3239-5cc8-4adf-8326-0e4ec2913235 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:11:15.378225 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1202/2412] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:11:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:11:15.379609 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6065f4e6-4515-44f9-ba90-bdcc0b8e8a32 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:11:15.389481 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6065f4e6-4515-44f9-ba90-bdcc0b8e8a32 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:11:15.394366 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f9384d-e502-4f3a-a2f4-51e08917f4e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:11:15.395001 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f9384d-e502-4f3a-a2f4-51e08917f4e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:11:15.395277 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f9384d-e502-4f3a-a2f4-51e08917f4e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:11:15.395543 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f9384d-e502-4f3a-a2f4-51e08917f4e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:11:15.396182 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f9384d-e502-4f3a-a2f4-51e08917f4e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:11:15.397844 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f9384d-e502-4f3a-a2f4-51e08917f4e0 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:11:15.412813 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1211/2413] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:11:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 11 12:11:15.414075 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ca3239-5cc8-4adf-8326-0e4ec2913235 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:11:15.427125 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ca3239-5cc8-4adf-8326-0e4ec2913235 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:11:15.431745 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ca97b0-6f42-40c4-b01a-f847586af8f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:11:15.432437 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ca97b0-6f42-40c4-b01a-f847586af8f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:11:15.432680 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ca97b0-6f42-40c4-b01a-f847586af8f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:11:15.432969 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ca97b0-6f42-40c4-b01a-f847586af8f9 admin admin] PATH_INFO: `/v3/projects/9867b89d69ac4bcd96135537313b1555/users/c46800376ffb410bb27d2c15c6d522b6/roles/de2d2bd26fff4748a79990f97a107164` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:11:15.445649 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:11:15.454194 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ca97b0-6f42-40c4-b01a-f847586af8f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9867b89d69ac4bcd96135537313b1555, user_id=c46800376ffb410bb27d2c15c6d522b6, role_id=de2d2bd26fff4748a79990f97a107164)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:11:15.457383 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ca97b0-6f42-40c4-b01a-f847586af8f9 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:11:15.474683 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1203/2414] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 12:11:15 2022] PUT /identity/v3/projects/9867b89d69ac4bcd96135537313b1555/users/c46800376ffb410bb27d2c15c6d522b6/roles/de2d2bd26fff4748a79990f97a107164 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 12:11:15.486254 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e56ab3a-8a8c-45b7-9a8c-e11b552e2009 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:11:15.486543 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e56ab3a-8a8c-45b7-9a8c-e11b552e2009 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:11:15.486850 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e56ab3a-8a8c-45b7-9a8c-e11b552e2009 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:11:15.514267 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-0e56ab3a-8a8c-45b7-9a8c-e11b552e2009 None None] MFA Rules not processed for user `c46800376ffb410bb27d2c15c6d522b6`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:11:15.515688 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-0e56ab3a-8a8c-45b7-9a8c-e11b552e2009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:11:15.579773 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:11:15.590047 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1212/2415] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:11:15 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:11:15.602208 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ca97b0-6f42-40c4-b01a-f847586af8f9 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:11:15.604165 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-44ca97b0-6f42-40c4-b01a-f847586af8f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:11:15.664728 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ca97b0-6f42-40c4-b01a-f847586af8f9 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:11:15.669310 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d4122e-17e2-4b05-b995-87b402b44b80 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91a00658b79a42bd91a22a2508b5de34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:11:15.669921 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d4122e-17e2-4b05-b995-87b402b44b80 service glance] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:11:15.670239 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d4122e-17e2-4b05-b995-87b402b44b80 service glance] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:11:15.670528 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d4122e-17e2-4b05-b995-87b402b44b80 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:11:15.681162 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d4122e-17e2-4b05-b995-87b402b44b80 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:11:15.682530 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d4122e-17e2-4b05-b995-87b402b44b80 service glance] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:11:15.691347 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1204/2416] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:11:15 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:11:18.822495 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e56ab3a-8a8c-45b7-9a8c-e11b552e2009 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:11:18.824179 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-0e56ab3a-8a8c-45b7-9a8c-e11b552e2009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:11:18.900494 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e56ab3a-8a8c-45b7-9a8c-e11b552e2009 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:11:18.905339 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9fafb2-ff5b-46bd-a13a-1b835ff34139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:11:18.905981 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9fafb2-ff5b-46bd-a13a-1b835ff34139 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:11:18.906258 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9fafb2-ff5b-46bd-a13a-1b835ff34139 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:11:18.906535 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9fafb2-ff5b-46bd-a13a-1b835ff34139 admin admin] PATH_INFO: `/v3/users/3434311c8240420896d6bfcb3428e515` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:11:18.907688 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9fafb2-ff5b-46bd-a13a-1b835ff34139 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3434311c8240420896d6bfcb3428e515)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:11:18.909641 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9fafb2-ff5b-46bd-a13a-1b835ff34139 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:11:18.966361 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-8f9fafb2-ff5b-46bd-a13a-1b835ff34139 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3434311c8240420896d6bfcb3428e515'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:11:18.981320 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-8f9fafb2-ff5b-46bd-a13a-1b835ff34139 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3434311c8240420896d6bfcb3428e515'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:11:18.982120 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-8f9fafb2-ff5b-46bd-a13a-1b835ff34139 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3434311c8240420896d6bfcb3428e515'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:11:18.993134 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-8f9fafb2-ff5b-46bd-a13a-1b835ff34139 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3434311c8240420896d6bfcb3428e515'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:11:18.993983 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-8f9fafb2-ff5b-46bd-a13a-1b835ff34139 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3434311c8240420896d6bfcb3428e515'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:11:19.010248 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1213/2417] 10.222.0.59 () {58 vars in 1224 bytes} [Tue Oct 11 12:11:18 2022] DELETE /identity/v3/users/3434311c8240420896d6bfcb3428e515 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:11:19.015057 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e875f02f-2190-4bde-89e4-0699f7a64c9b None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:11:19.015569 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e875f02f-2190-4bde-89e4-0699f7a64c9b None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:11:19.015569 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e875f02f-2190-4bde-89e4-0699f7a64c9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:11:19.036610 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-e875f02f-2190-4bde-89e4-0699f7a64c9b None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:11:19.037844 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-e875f02f-2190-4bde-89e4-0699f7a64c9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:11:19.102667 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1205/2418] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:11:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:11:19.113404 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9fafb2-ff5b-46bd-a13a-1b835ff34139 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:11:19.115081 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-8f9fafb2-ff5b-46bd-a13a-1b835ff34139 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:11:19.183004 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9fafb2-ff5b-46bd-a13a-1b835ff34139 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:11:19.186997 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1063aa6d-66dd-412e-a78a-6ebc622681dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e563fe174db4cc19ae6bf8162cc3d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:11:19.187684 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1063aa6d-66dd-412e-a78a-6ebc622681dc service neutron] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:11:19.187971 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1063aa6d-66dd-412e-a78a-6ebc622681dc service neutron] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:11:19.188232 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1063aa6d-66dd-412e-a78a-6ebc622681dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:11:19.199426 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1063aa6d-66dd-412e-a78a-6ebc622681dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:11:19.201588 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1063aa6d-66dd-412e-a78a-6ebc622681dc service neutron] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:11:19.215269 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1214/2419] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:11:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:11:19.486032 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e875f02f-2190-4bde-89e4-0699f7a64c9b None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:11:19.487629 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-e875f02f-2190-4bde-89e4-0699f7a64c9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:11:19.502944 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e875f02f-2190-4bde-89e4-0699f7a64c9b None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:11:19.509202 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48a1865-2d59-41df-9fb6-e18e7cee57b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:11:19.509876 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48a1865-2d59-41df-9fb6-e18e7cee57b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:11:19.510124 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48a1865-2d59-41df-9fb6-e18e7cee57b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:11:19.510397 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48a1865-2d59-41df-9fb6-e18e7cee57b3 admin admin] PATH_INFO: `/v3/projects/6e42ea3c037b44b1a6dc2ca0a9ffdd6c` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:11:19.511706 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48a1865-2d59-41df-9fb6-e18e7cee57b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e42ea3c037b44b1a6dc2ca0a9ffdd6c)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:11:19.513654 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48a1865-2d59-41df-9fb6-e18e7cee57b3 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:11:19.563398 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-a48a1865-2d59-41df-9fb6-e18e7cee57b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e42ea3c037b44b1a6dc2ca0a9ffdd6c', 'request_id': 'req-a48a1865-2d59-41df-9fb6-e18e7cee57b3'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:11:19.569141 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-a48a1865-2d59-41df-9fb6-e18e7cee57b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e42ea3c037b44b1a6dc2ca0a9ffdd6c', 'request_id': 'req-a48a1865-2d59-41df-9fb6-e18e7cee57b3'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:11:19.577128 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-a48a1865-2d59-41df-9fb6-e18e7cee57b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e42ea3c037b44b1a6dc2ca0a9ffdd6c', 'request_id': 'req-a48a1865-2d59-41df-9fb6-e18e7cee57b3'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:11:19.588626 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-a48a1865-2d59-41df-9fb6-e18e7cee57b3 admin admin] The token cache is being invalidate because project 6e42ea3c037b44b1a6dc2ca0a9ffdd6c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 12:11:19.588917 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-a48a1865-2d59-41df-9fb6-e18e7cee57b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:11:19.589589 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-a48a1865-2d59-41df-9fb6-e18e7cee57b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:11:19.591180 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1206/2420] 10.222.0.59 () {58 vars in 1233 bytes} [Tue Oct 11 12:11:19 2022] DELETE /identity/v3/projects/6e42ea3c037b44b1a6dc2ca0a9ffdd6c => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:11:19.596520 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aa95e3-9d9f-4096-876e-8848b50be6c8 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:11:19.596805 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aa95e3-9d9f-4096-876e-8848b50be6c8 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:11:19.597103 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aa95e3-9d9f-4096-876e-8848b50be6c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:11:19.618626 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-02aa95e3-9d9f-4096-876e-8848b50be6c8 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:11:19.619762 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-02aa95e3-9d9f-4096-876e-8848b50be6c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:11:19.632291 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:11:19.688180 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1215/2421] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:11:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:11:19.696589 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ca59f5-b5d0-4b23-80ad-2bb4c6ff4aae None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:11:19.696879 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ca59f5-b5d0-4b23-80ad-2bb4c6ff4aae None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:11:19.697156 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ca59f5-b5d0-4b23-80ad-2bb4c6ff4aae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:11:19.718596 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-31ca59f5-b5d0-4b23-80ad-2bb4c6ff4aae None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:11:19.719850 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-31ca59f5-b5d0-4b23-80ad-2bb4c6ff4aae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:11:19.728566 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1207/2422] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:11:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:11:19.733274 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02aa95e3-9d9f-4096-876e-8848b50be6c8 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:11:19.744068 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02aa95e3-9d9f-4096-876e-8848b50be6c8 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:11:19.750548 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c694a60a-784c-4ed8-b80e-5363e4adb640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:11:19.751188 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c694a60a-784c-4ed8-b80e-5363e4adb640 admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:11:19.751468 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c694a60a-784c-4ed8-b80e-5363e4adb640 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:11:19.751767 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c694a60a-784c-4ed8-b80e-5363e4adb640 admin admin] PATH_INFO: `/v3/domains` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:11:19.752442 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c694a60a-784c-4ed8-b80e-5363e4adb640 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 12:11:19.752747 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c694a60a-784c-4ed8-b80e-5363e4adb640 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:11:19.754519 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c694a60a-784c-4ed8-b80e-5363e4adb640 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:11:19.769074 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1216/2423] 10.222.0.59 () {58 vars in 1153 bytes} [Tue Oct 11 12:11:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 12:11:19.775223 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ca59f5-b5d0-4b23-80ad-2bb4c6ff4aae None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:11:19.789269 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ca59f5-b5d0-4b23-80ad-2bb4c6ff4aae None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:11:19.796462 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25d5b29-8382-4e3c-89f4-160864c5dab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:11:19.797131 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25d5b29-8382-4e3c-89f4-160864c5dab1 admin admin] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:11:19.797460 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25d5b29-8382-4e3c-89f4-160864c5dab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:11:19.797825 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25d5b29-8382-4e3c-89f4-160864c5dab1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:11:19.798727 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25d5b29-8382-4e3c-89f4-160864c5dab1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:11:19.800755 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25d5b29-8382-4e3c-89f4-160864c5dab1 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:11:19.805827 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:11:19.833265 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1208/2424] 10.222.0.59 () {60 vars in 1153 bytes} [Tue Oct 11 12:11:19 2022] POST /identity/v3/projects => generated 383 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:11:19.837901 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c694a60a-784c-4ed8-b80e-5363e4adb640 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:11:19.850408 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c694a60a-784c-4ed8-b80e-5363e4adb640 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:11:19.913450 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766b8d79-16c8-4315-a395-0f3eb7e3387f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:11:19.913966 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766b8d79-16c8-4315-a395-0f3eb7e3387f admin admin] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:11:19.914239 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766b8d79-16c8-4315-a395-0f3eb7e3387f admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:11:19.914511 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766b8d79-16c8-4315-a395-0f3eb7e3387f admin admin] PATH_INFO: `/v3/users` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:11:19.915168 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766b8d79-16c8-4315-a395-0f3eb7e3387f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:11:19.917141 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766b8d79-16c8-4315-a395-0f3eb7e3387f admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:11:19.941640 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1217/2425] 10.222.0.59 () {60 vars in 1144 bytes} [Tue Oct 11 12:11:19 2022] POST /identity/v3/users => generated 292 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:11:19.947222 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25d5b29-8382-4e3c-89f4-160864c5dab1 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:11:19.958196 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25d5b29-8382-4e3c-89f4-160864c5dab1 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:11:19.963862 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02c5ddb-8e6d-45e3-bfa0-6a72ccee7594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:11:19.964474 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02c5ddb-8e6d-45e3-bfa0-6a72ccee7594 admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:11:19.964749 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02c5ddb-8e6d-45e3-bfa0-6a72ccee7594 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:11:19.965020 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02c5ddb-8e6d-45e3-bfa0-6a72ccee7594 admin admin] PATH_INFO: `/v3/roles` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:11:19.966138 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02c5ddb-8e6d-45e3-bfa0-6a72ccee7594 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:11:19.967341 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02c5ddb-8e6d-45e3-bfa0-6a72ccee7594 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:11:19.978205 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1209/2426] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:11:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:11:19.982703 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766b8d79-16c8-4315-a395-0f3eb7e3387f admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:11:19.994075 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766b8d79-16c8-4315-a395-0f3eb7e3387f admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:11:20.000484 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b41c15-5232-4af8-9ae8-061642bc9c07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:11:20.001074 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b41c15-5232-4af8-9ae8-061642bc9c07 admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:11:20.001356 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b41c15-5232-4af8-9ae8-061642bc9c07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:11:20.001659 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b41c15-5232-4af8-9ae8-061642bc9c07 admin admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:11:20.002315 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b41c15-5232-4af8-9ae8-061642bc9c07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:11:20.003982 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b41c15-5232-4af8-9ae8-061642bc9c07 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:11:20.014986 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1218/2427] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:11:19 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:11:20.019987 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02c5ddb-8e6d-45e3-bfa0-6a72ccee7594 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:11:20.030127 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02c5ddb-8e6d-45e3-bfa0-6a72ccee7594 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:11:20.035277 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c455df49-2b25-4ac6-83c0-06427e9effc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:11:20.035932 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c455df49-2b25-4ac6-83c0-06427e9effc3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:11:20.036212 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c455df49-2b25-4ac6-83c0-06427e9effc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:11:20.036485 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c455df49-2b25-4ac6-83c0-06427e9effc3 admin admin] PATH_INFO: `/v3/projects/f48db192fac04a06932590c23b751595/users/33ade5d60ccb44bab2e8ee2f0b21e456/roles/de2d2bd26fff4748a79990f97a107164` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:11:20.056287 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c455df49-2b25-4ac6-83c0-06427e9effc3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f48db192fac04a06932590c23b751595, user_id=33ade5d60ccb44bab2e8ee2f0b21e456, role_id=de2d2bd26fff4748a79990f97a107164)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:11:20.059003 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c455df49-2b25-4ac6-83c0-06427e9effc3 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:11:20.074138 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1210/2428] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 12:11:20 2022] PUT /identity/v3/projects/f48db192fac04a06932590c23b751595/users/33ade5d60ccb44bab2e8ee2f0b21e456/roles/de2d2bd26fff4748a79990f97a107164 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 12:11:20.079525 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c3eed8-598c-4ae8-bbec-1b8daa578dac None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:11:20.079819 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c3eed8-598c-4ae8-bbec-1b8daa578dac None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:11:20.080092 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c3eed8-598c-4ae8-bbec-1b8daa578dac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:11:20.099710 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-d3c3eed8-598c-4ae8-bbec-1b8daa578dac None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:11:20.100948 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-d3c3eed8-598c-4ae8-bbec-1b8daa578dac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:11:20.170734 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1219/2429] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:11:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:11:20.181734 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c455df49-2b25-4ac6-83c0-06427e9effc3 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:11:20.183613 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-c455df49-2b25-4ac6-83c0-06427e9effc3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:11:20.249386 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c455df49-2b25-4ac6-83c0-06427e9effc3 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:11:20.253329 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b04008-14da-42ff-b437-4f88a51ce718 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e563fe174db4cc19ae6bf8162cc3d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:11:20.254045 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b04008-14da-42ff-b437-4f88a51ce718 service neutron] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:11:20.254314 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b04008-14da-42ff-b437-4f88a51ce718 service neutron] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:11:20.254607 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b04008-14da-42ff-b437-4f88a51ce718 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:11:20.264659 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b04008-14da-42ff-b437-4f88a51ce718 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:11:20.266165 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b04008-14da-42ff-b437-4f88a51ce718 service neutron] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:11:20.276779 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1211/2430] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:11:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:11:24.666103 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7587c5-a23a-4aad-8cfc-24f0a1ea7990 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:11:24.666714 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7587c5-a23a-4aad-8cfc-24f0a1ea7990 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:11:24.666714 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7587c5-a23a-4aad-8cfc-24f0a1ea7990 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:11:24.707436 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-8f7587c5-a23a-4aad-8cfc-24f0a1ea7990 None None] MFA Rules not processed for user `33ade5d60ccb44bab2e8ee2f0b21e456`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:11:24.711421 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-8f7587c5-a23a-4aad-8cfc-24f0a1ea7990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:11:24.733199 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:11:24.796743 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1220/2431] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:11:24 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:11:24.801628 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b04008-14da-42ff-b437-4f88a51ce718 service neutron] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:11:24.823043 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b04008-14da-42ff-b437-4f88a51ce718 service neutron] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:11:24.843244 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff39873b-3380-4242-a8d0-dad2bc9b7e02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:11:24.851420 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff39873b-3380-4242-a8d0-dad2bc9b7e02 admin admin] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:11:24.851709 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff39873b-3380-4242-a8d0-dad2bc9b7e02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:11:24.851983 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff39873b-3380-4242-a8d0-dad2bc9b7e02 admin admin] PATH_INFO: `/v3/projects` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:11:24.852696 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff39873b-3380-4242-a8d0-dad2bc9b7e02 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:11:24.854678 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff39873b-3380-4242-a8d0-dad2bc9b7e02 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:11:24.891123 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1212/2432] 10.222.0.59 () {60 vars in 1153 bytes} [Tue Oct 11 12:11:24 2022] POST /identity/v3/projects => generated 385 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 12:11:24.895573 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7587c5-a23a-4aad-8cfc-24f0a1ea7990 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:11:24.909520 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7587c5-a23a-4aad-8cfc-24f0a1ea7990 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:11:24.978865 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f9f231-2474-46db-b57c-c41de03bcf77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:11:24.983704 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f9f231-2474-46db-b57c-c41de03bcf77 admin admin] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:11:24.983704 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f9f231-2474-46db-b57c-c41de03bcf77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:11:24.983704 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f9f231-2474-46db-b57c-c41de03bcf77 admin admin] PATH_INFO: `/v3/users` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:11:24.983704 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f9f231-2474-46db-b57c-c41de03bcf77 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:11:24.983704 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f9f231-2474-46db-b57c-c41de03bcf77 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:11:25.007363 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1221/2433] 10.222.0.59 () {60 vars in 1144 bytes} [Tue Oct 11 12:11:24 2022] POST /identity/v3/users => generated 299 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:11:25.011705 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff39873b-3380-4242-a8d0-dad2bc9b7e02 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:11:25.023215 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff39873b-3380-4242-a8d0-dad2bc9b7e02 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:11:25.031679 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9c1f9a-95c3-48b3-9841-64710ecf5762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:11:25.032332 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9c1f9a-95c3-48b3-9841-64710ecf5762 admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:11:25.032613 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9c1f9a-95c3-48b3-9841-64710ecf5762 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:11:25.032882 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9c1f9a-95c3-48b3-9841-64710ecf5762 admin admin] PATH_INFO: `/v3/roles` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:11:25.033541 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9c1f9a-95c3-48b3-9841-64710ecf5762 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:11:25.035433 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9c1f9a-95c3-48b3-9841-64710ecf5762 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:11:25.047208 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1213/2434] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:11:25 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:11:25.051602 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f9f231-2474-46db-b57c-c41de03bcf77 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:11:25.062742 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f9f231-2474-46db-b57c-c41de03bcf77 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:11:25.068039 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a771c9-e3c3-4310-8278-20bc77470571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:11:25.068772 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a771c9-e3c3-4310-8278-20bc77470571 admin admin] REQUEST_METHOD: `PUT` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:11:25.069055 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a771c9-e3c3-4310-8278-20bc77470571 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:11:25.069329 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a771c9-e3c3-4310-8278-20bc77470571 admin admin] PATH_INFO: `/v3/projects/a6fbe903c2b545ca997d121e213f4611/users/cf8bd4dbc2f144aeaadeb8a819c5d2c4/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:11:25.085885 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a771c9-e3c3-4310-8278-20bc77470571 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6fbe903c2b545ca997d121e213f4611, user_id=cf8bd4dbc2f144aeaadeb8a819c5d2c4, role_id=5cf1f4bd2fef46ab9c6eff56d52f9f69)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:11:25.088512 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a771c9-e3c3-4310-8278-20bc77470571 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:11:25.103649 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1222/2435] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 12:11:25 2022] PUT /identity/v3/projects/a6fbe903c2b545ca997d121e213f4611/users/cf8bd4dbc2f144aeaadeb8a819c5d2c4/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:11:25.107994 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9c1f9a-95c3-48b3-9841-64710ecf5762 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:11:25.119004 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9c1f9a-95c3-48b3-9841-64710ecf5762 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:11:25.188766 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f0d0f4-edc4-4b02-acee-30390512d029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:11:25.189408 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f0d0f4-edc4-4b02-acee-30390512d029 admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:11:25.189768 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f0d0f4-edc4-4b02-acee-30390512d029 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:11:25.190086 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f0d0f4-edc4-4b02-acee-30390512d029 admin admin] PATH_INFO: `/v3/roles` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:11:25.190938 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f0d0f4-edc4-4b02-acee-30390512d029 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:11:25.192707 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f0d0f4-edc4-4b02-acee-30390512d029 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:11:25.205028 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1214/2436] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:11:25 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:11:25.209372 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a771c9-e3c3-4310-8278-20bc77470571 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:11:25.222096 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a771c9-e3c3-4310-8278-20bc77470571 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:11:25.228219 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde78291-207c-46ed-a623-3c54cf2b65ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:11:25.229182 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde78291-207c-46ed-a623-3c54cf2b65ba admin admin] REQUEST_METHOD: `PUT` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:11:25.229584 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde78291-207c-46ed-a623-3c54cf2b65ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:11:25.229899 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde78291-207c-46ed-a623-3c54cf2b65ba admin admin] PATH_INFO: `/v3/projects/a6fbe903c2b545ca997d121e213f4611/users/cf8bd4dbc2f144aeaadeb8a819c5d2c4/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:11:25.231947 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde78291-207c-46ed-a623-3c54cf2b65ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6fbe903c2b545ca997d121e213f4611, user_id=cf8bd4dbc2f144aeaadeb8a819c5d2c4, role_id=5cf1f4bd2fef46ab9c6eff56d52f9f69)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:11:25.234683 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde78291-207c-46ed-a623-3c54cf2b65ba admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:11:25.246140 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1223/2437] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 12:11:25 2022] PUT /identity/v3/projects/a6fbe903c2b545ca997d121e213f4611/users/cf8bd4dbc2f144aeaadeb8a819c5d2c4/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 12:11:29.638913 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d834fa-23e9-4476-bb5b-c3ef418c34f4 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:11:29.639477 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d834fa-23e9-4476-bb5b-c3ef418c34f4 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:11:29.639477 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d834fa-23e9-4476-bb5b-c3ef418c34f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:11:29.672138 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-d7d834fa-23e9-4476-bb5b-c3ef418c34f4 None None] MFA Rules not processed for user `cf8bd4dbc2f144aeaadeb8a819c5d2c4`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:11:29.673892 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-d7d834fa-23e9-4476-bb5b-c3ef418c34f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:11:29.678111 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:11:29.748074 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1215/2438] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:11:29 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:11:29.761511 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde78291-207c-46ed-a623-3c54cf2b65ba admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:11:29.763572 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-cde78291-207c-46ed-a623-3c54cf2b65ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:11:29.830344 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde78291-207c-46ed-a623-3c54cf2b65ba admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:11:29.834359 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d951a0-1882-4998-aa74-b45304b9b4de service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91a00658b79a42bd91a22a2508b5de34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:11:29.835011 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d951a0-1882-4998-aa74-b45304b9b4de service glance] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:11:29.835282 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d951a0-1882-4998-aa74-b45304b9b4de service glance] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:11:29.835552 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d951a0-1882-4998-aa74-b45304b9b4de service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:11:29.856213 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:11:29.901352 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d951a0-1882-4998-aa74-b45304b9b4de service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:11:29.902963 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d951a0-1882-4998-aa74-b45304b9b4de service glance] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:11:29.913399 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1224/2439] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:11:29 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:11:30.096914 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8f83b0-65e9-4c87-9a54-d77709719d1f None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:11:30.097250 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8f83b0-65e9-4c87-9a54-d77709719d1f None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:11:30.097538 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8f83b0-65e9-4c87-9a54-d77709719d1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:11:30.118369 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-7f8f83b0-65e9-4c87-9a54-d77709719d1f None None] MFA Rules not processed for user `8a19eceb62ce4157a4c5fe1866a89df9`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:11:30.119625 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-7f8f83b0-65e9-4c87-9a54-d77709719d1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:11:30.190786 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1216/2440] 10.222.0.59 () {58 vars in 1009 bytes} [Tue Oct 11 12:11:30 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 12:11:30.214433 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d951a0-1882-4998-aa74-b45304b9b4de service glance] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:11:30.218829 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-73d951a0-1882-4998-aa74-b45304b9b4de service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:11:30.306297 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d951a0-1882-4998-aa74-b45304b9b4de service glance] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:11:30.312468 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7674fe-9647-4775-b20e-6759d93d2258 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76e7fee1993a40d5ba4295e7daeb3e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:11:30.313120 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7674fe-9647-4775-b20e-6759d93d2258 service swift] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:11:30.313394 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7674fe-9647-4775-b20e-6759d93d2258 service swift] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:11:30.313671 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7674fe-9647-4775-b20e-6759d93d2258 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:11:30.323614 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7674fe-9647-4775-b20e-6759d93d2258 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:11:30.325249 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7674fe-9647-4775-b20e-6759d93d2258 service swift] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:11:30.334961 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1225/2441] 10.222.0.59 () {60 vars in 1384 bytes} [Tue Oct 11 12:11:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 11 12:11:39.114336 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad8d2e9-a874-4140-9656-db59c8ab3944 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:11:39.114936 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad8d2e9-a874-4140-9656-db59c8ab3944 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:11:39.114936 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad8d2e9-a874-4140-9656-db59c8ab3944 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:11:39.135102 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-fad8d2e9-a874-4140-9656-db59c8ab3944 None None] MFA Rules not processed for user `33ade5d60ccb44bab2e8ee2f0b21e456`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:11:39.136352 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-fad8d2e9-a874-4140-9656-db59c8ab3944 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:11:39.143970 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1217/2442] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:11:39 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:11:39.441177 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7674fe-9647-4775-b20e-6759d93d2258 service swift] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:11:39.500504 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7674fe-9647-4775-b20e-6759d93d2258 service swift] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:11:39.504476 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4536c8f-271c-4b64-a3e3-8532d784ef82 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e563fe174db4cc19ae6bf8162cc3d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:11:39.505110 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4536c8f-271c-4b64-a3e3-8532d784ef82 service neutron] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:11:39.505384 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4536c8f-271c-4b64-a3e3-8532d784ef82 service neutron] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:11:39.505653 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4536c8f-271c-4b64-a3e3-8532d784ef82 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:11:39.507535 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-e4536c8f-271c-4b64-a3e3-8532d784ef82 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:11:39.519276 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4536c8f-271c-4b64-a3e3-8532d784ef82 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:11:39.520796 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4536c8f-271c-4b64-a3e3-8532d784ef82 service neutron] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:11:39.529246 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1226/2443] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:11:39 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:11:39.896707 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad8d2e9-a874-4140-9656-db59c8ab3944 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:11:39.898236 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-fad8d2e9-a874-4140-9656-db59c8ab3944 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:11:39.985975 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad8d2e9-a874-4140-9656-db59c8ab3944 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:11:39.991144 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2fd087-b5df-4a50-9e6b-837302e8cc22 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f272cdd1bcc04b4e8261ed533cfd4fd7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:11:39.991756 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2fd087-b5df-4a50-9e6b-837302e8cc22 service placement] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:11:39.992031 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2fd087-b5df-4a50-9e6b-837302e8cc22 service placement] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:11:39.992308 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2fd087-b5df-4a50-9e6b-837302e8cc22 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:11:39.994377 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-7c2fd087-b5df-4a50-9e6b-837302e8cc22 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:11:40.006132 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2fd087-b5df-4a50-9e6b-837302e8cc22 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:11:40.007682 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2fd087-b5df-4a50-9e6b-837302e8cc22 service placement] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:11:40.020083 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1218/2444] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:11:39 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:11:41.200462 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4536c8f-271c-4b64-a3e3-8532d784ef82 service neutron] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:11:41.202065 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-e4536c8f-271c-4b64-a3e3-8532d784ef82 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:11:41.251875 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:11:41.278400 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4536c8f-271c-4b64-a3e3-8532d784ef82 service neutron] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:11:41.283336 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c144a8-69c8-44a2-8d2d-a18cfc35df27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:11:41.283998 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c144a8-69c8-44a2-8d2d-a18cfc35df27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:11:41.284281 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c144a8-69c8-44a2-8d2d-a18cfc35df27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:11:41.284557 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c144a8-69c8-44a2-8d2d-a18cfc35df27 admin admin] PATH_INFO: `/v3/users/c46800376ffb410bb27d2c15c6d522b6` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:11:41.285767 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c144a8-69c8-44a2-8d2d-a18cfc35df27 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c46800376ffb410bb27d2c15c6d522b6)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:11:41.287839 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c144a8-69c8-44a2-8d2d-a18cfc35df27 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:11:41.342507 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-20c144a8-69c8-44a2-8d2d-a18cfc35df27 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c46800376ffb410bb27d2c15c6d522b6'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:11:41.354909 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-20c144a8-69c8-44a2-8d2d-a18cfc35df27 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c46800376ffb410bb27d2c15c6d522b6'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:11:41.355711 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-20c144a8-69c8-44a2-8d2d-a18cfc35df27 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c46800376ffb410bb27d2c15c6d522b6'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:11:41.366309 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-20c144a8-69c8-44a2-8d2d-a18cfc35df27 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c46800376ffb410bb27d2c15c6d522b6'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:11:41.367263 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-20c144a8-69c8-44a2-8d2d-a18cfc35df27 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c46800376ffb410bb27d2c15c6d522b6'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:11:41.384832 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1227/2445] 10.222.0.59 () {58 vars in 1224 bytes} [Tue Oct 11 12:11:41 2022] DELETE /identity/v3/users/c46800376ffb410bb27d2c15c6d522b6 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:11:41.389190 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7969bdec-2118-4f7e-852b-4c947628f98e None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:11:41.390624 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7969bdec-2118-4f7e-852b-4c947628f98e None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:11:41.390889 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7969bdec-2118-4f7e-852b-4c947628f98e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:11:41.412663 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-7969bdec-2118-4f7e-852b-4c947628f98e None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:11:41.413966 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-7969bdec-2118-4f7e-852b-4c947628f98e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:11:41.438659 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:11:41.479745 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1219/2446] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:11:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:11:41.487637 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c144a8-69c8-44a2-8d2d-a18cfc35df27 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:11:41.489211 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-20c144a8-69c8-44a2-8d2d-a18cfc35df27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:11:41.530744 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b505ff-2acb-488e-b630-8d179fef366c None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:11:41.531978 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b505ff-2acb-488e-b630-8d179fef366c None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:11:41.532328 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b505ff-2acb-488e-b630-8d179fef366c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:11:41.554776 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-92b505ff-2acb-488e-b630-8d179fef366c None None] MFA Rules not processed for user `8a19eceb62ce4157a4c5fe1866a89df9`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:11:41.556187 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-92b505ff-2acb-488e-b630-8d179fef366c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:11:41.557708 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c144a8-69c8-44a2-8d2d-a18cfc35df27 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:11:41.562723 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3834fd27-c59c-4e15-8b8a-dcbcf6030a43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e563fe174db4cc19ae6bf8162cc3d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:11:41.563421 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3834fd27-c59c-4e15-8b8a-dcbcf6030a43 service neutron] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:11:41.563631 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3834fd27-c59c-4e15-8b8a-dcbcf6030a43 service neutron] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:11:41.563956 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3834fd27-c59c-4e15-8b8a-dcbcf6030a43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:11:41.575094 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3834fd27-c59c-4e15-8b8a-dcbcf6030a43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:11:41.576689 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3834fd27-c59c-4e15-8b8a-dcbcf6030a43 service neutron] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:11:41.589456 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1228/2447] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:11:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:11:41.617650 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1220/2448] 10.222.0.59 () {58 vars in 1009 bytes} [Tue Oct 11 12:11:41 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 12:11:41.630478 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3834fd27-c59c-4e15-8b8a-dcbcf6030a43 service neutron] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:11:41.632048 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-3834fd27-c59c-4e15-8b8a-dcbcf6030a43 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:11:41.709783 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3834fd27-c59c-4e15-8b8a-dcbcf6030a43 service neutron] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:11:41.716615 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a11b7ca-37fa-4723-82da-a571203eecf0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76e7fee1993a40d5ba4295e7daeb3e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:11:41.717245 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a11b7ca-37fa-4723-82da-a571203eecf0 service swift] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:11:41.717523 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a11b7ca-37fa-4723-82da-a571203eecf0 service swift] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:11:41.717795 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a11b7ca-37fa-4723-82da-a571203eecf0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:11:41.723515 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:11:41.728679 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a11b7ca-37fa-4723-82da-a571203eecf0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:11:41.730968 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a11b7ca-37fa-4723-82da-a571203eecf0 service swift] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:11:41.742321 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1229/2449] 10.222.0.59 () {60 vars in 1384 bytes} [Tue Oct 11 12:11:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 11 12:11:42.226390 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b505ff-2acb-488e-b630-8d179fef366c None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:11:42.228078 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-92b505ff-2acb-488e-b630-8d179fef366c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:11:42.255086 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b505ff-2acb-488e-b630-8d179fef366c None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:11:42.268074 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb6d2b2-94a4-43ad-9b59-d125cd4f9e2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:11:42.268903 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb6d2b2-94a4-43ad-9b59-d125cd4f9e2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:11:42.269143 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb6d2b2-94a4-43ad-9b59-d125cd4f9e2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:11:42.269483 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb6d2b2-94a4-43ad-9b59-d125cd4f9e2c admin admin] PATH_INFO: `/v3/projects/9867b89d69ac4bcd96135537313b1555` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:11:42.279260 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb6d2b2-94a4-43ad-9b59-d125cd4f9e2c admin admin] RBAC: Authorizing `identity:delete_project(project_id=9867b89d69ac4bcd96135537313b1555)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:11:42.281720 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb6d2b2-94a4-43ad-9b59-d125cd4f9e2c admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:11:42.381765 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-feb6d2b2-94a4-43ad-9b59-d125cd4f9e2c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9867b89d69ac4bcd96135537313b1555', 'request_id': 'req-feb6d2b2-94a4-43ad-9b59-d125cd4f9e2c'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:11:42.392208 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-feb6d2b2-94a4-43ad-9b59-d125cd4f9e2c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9867b89d69ac4bcd96135537313b1555', 'request_id': 'req-feb6d2b2-94a4-43ad-9b59-d125cd4f9e2c'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:11:42.403318 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-feb6d2b2-94a4-43ad-9b59-d125cd4f9e2c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9867b89d69ac4bcd96135537313b1555', 'request_id': 'req-feb6d2b2-94a4-43ad-9b59-d125cd4f9e2c'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:11:42.415795 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-feb6d2b2-94a4-43ad-9b59-d125cd4f9e2c admin admin] The token cache is being invalidate because project 9867b89d69ac4bcd96135537313b1555 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 12:11:42.416099 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-feb6d2b2-94a4-43ad-9b59-d125cd4f9e2c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:11:42.416767 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-feb6d2b2-94a4-43ad-9b59-d125cd4f9e2c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:11:42.421806 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1221/2450] 10.222.0.59 () {58 vars in 1233 bytes} [Tue Oct 11 12:11:42 2022] DELETE /identity/v3/projects/9867b89d69ac4bcd96135537313b1555 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:11:42.423845 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06afafc0-5182-4fc6-967b-fdd12051ebae None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:11:42.424351 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06afafc0-5182-4fc6-967b-fdd12051ebae None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:11:42.424453 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06afafc0-5182-4fc6-967b-fdd12051ebae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:11:42.445084 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-06afafc0-5182-4fc6-967b-fdd12051ebae None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:11:42.449911 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-06afafc0-5182-4fc6-967b-fdd12051ebae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:11:42.539353 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1230/2451] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:11:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 12:11:42.541722 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6a0e07-760e-481d-a2ef-6c151410d7b2 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:11:42.542004 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6a0e07-760e-481d-a2ef-6c151410d7b2 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:11:42.542441 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6a0e07-760e-481d-a2ef-6c151410d7b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:11:42.563636 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-ce6a0e07-760e-481d-a2ef-6c151410d7b2 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:11:42.564882 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-ce6a0e07-760e-481d-a2ef-6c151410d7b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:11:42.579409 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1222/2452] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:11:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:11:42.580828 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06afafc0-5182-4fc6-967b-fdd12051ebae None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:11:42.593507 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06afafc0-5182-4fc6-967b-fdd12051ebae None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:11:42.601176 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44419baf-51e4-4164-ac78-e73fdc55f1e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:11:42.602315 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44419baf-51e4-4164-ac78-e73fdc55f1e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:11:42.602789 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44419baf-51e4-4164-ac78-e73fdc55f1e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:11:42.603170 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44419baf-51e4-4164-ac78-e73fdc55f1e7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:11:42.603947 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44419baf-51e4-4164-ac78-e73fdc55f1e7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 12:11:42.604231 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44419baf-51e4-4164-ac78-e73fdc55f1e7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:11:42.606063 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44419baf-51e4-4164-ac78-e73fdc55f1e7 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:11:42.620141 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1231/2453] 10.222.0.59 () {58 vars in 1153 bytes} [Tue Oct 11 12:11:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 12:11:42.624617 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6a0e07-760e-481d-a2ef-6c151410d7b2 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:11:42.634509 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6a0e07-760e-481d-a2ef-6c151410d7b2 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:11:42.639250 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c195cb42-4728-435d-bc42-f1e53ac38d2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:11:42.639898 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c195cb42-4728-435d-bc42-f1e53ac38d2f admin admin] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:11:42.640178 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c195cb42-4728-435d-bc42-f1e53ac38d2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:11:42.640453 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c195cb42-4728-435d-bc42-f1e53ac38d2f admin admin] PATH_INFO: `/v3/projects` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:11:42.641113 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c195cb42-4728-435d-bc42-f1e53ac38d2f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:11:42.643450 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c195cb42-4728-435d-bc42-f1e53ac38d2f admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:11:42.679210 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1223/2454] 10.222.0.59 () {60 vars in 1153 bytes} [Tue Oct 11 12:11:42 2022] POST /identity/v3/projects => generated 365 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:11:42.684364 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44419baf-51e4-4164-ac78-e73fdc55f1e7 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:11:42.696756 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44419baf-51e4-4164-ac78-e73fdc55f1e7 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:11:42.768827 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18554048-c2e5-426d-b27e-954e8f66a7da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:11:42.769955 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18554048-c2e5-426d-b27e-954e8f66a7da admin admin] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:11:42.770415 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18554048-c2e5-426d-b27e-954e8f66a7da admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:11:42.770999 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18554048-c2e5-426d-b27e-954e8f66a7da admin admin] PATH_INFO: `/v3/users` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:11:42.772128 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18554048-c2e5-426d-b27e-954e8f66a7da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:11:42.774999 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18554048-c2e5-426d-b27e-954e8f66a7da admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:11:42.799788 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1232/2455] 10.222.0.59 () {60 vars in 1144 bytes} [Tue Oct 11 12:11:42 2022] POST /identity/v3/users => generated 283 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 12:11:42.804159 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c195cb42-4728-435d-bc42-f1e53ac38d2f admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:11:42.814364 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c195cb42-4728-435d-bc42-f1e53ac38d2f admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:11:42.819588 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f9bf24-5963-4e90-b983-11691dd6005c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:11:42.820342 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f9bf24-5963-4e90-b983-11691dd6005c admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:11:42.820689 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f9bf24-5963-4e90-b983-11691dd6005c admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:11:42.820996 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f9bf24-5963-4e90-b983-11691dd6005c admin admin] PATH_INFO: `/v3/roles` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:11:42.821662 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f9bf24-5963-4e90-b983-11691dd6005c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:11:42.823447 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f9bf24-5963-4e90-b983-11691dd6005c admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:11:42.835185 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1224/2456] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:11:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:11:42.839142 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18554048-c2e5-426d-b27e-954e8f66a7da admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:11:42.852104 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18554048-c2e5-426d-b27e-954e8f66a7da admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:11:42.858253 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6284ce-315d-4ef8-9daf-745c29fd4310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:11:42.858993 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6284ce-315d-4ef8-9daf-745c29fd4310 admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:11:42.859270 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6284ce-315d-4ef8-9daf-745c29fd4310 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:11:42.859545 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6284ce-315d-4ef8-9daf-745c29fd4310 admin admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:11:42.860198 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6284ce-315d-4ef8-9daf-745c29fd4310 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:11:42.862128 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6284ce-315d-4ef8-9daf-745c29fd4310 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:11:42.874240 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1233/2457] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:11:42 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 11 12:11:42.878463 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f9bf24-5963-4e90-b983-11691dd6005c admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:11:42.889139 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f9bf24-5963-4e90-b983-11691dd6005c admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:11:42.894118 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d4ffd6-6f03-4a9d-91f5-f6983b82587f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:11:42.894972 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d4ffd6-6f03-4a9d-91f5-f6983b82587f admin admin] REQUEST_METHOD: `PUT` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:11:42.895256 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d4ffd6-6f03-4a9d-91f5-f6983b82587f admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:11:42.895529 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d4ffd6-6f03-4a9d-91f5-f6983b82587f admin admin] PATH_INFO: `/v3/projects/bc8a1a7d21b245278a7d663ac4d78129/users/586dd2a7012d4ad1841e837eeecfd247/roles/de2d2bd26fff4748a79990f97a107164` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:11:42.912192 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d4ffd6-6f03-4a9d-91f5-f6983b82587f admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc8a1a7d21b245278a7d663ac4d78129, user_id=586dd2a7012d4ad1841e837eeecfd247, role_id=de2d2bd26fff4748a79990f97a107164)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:11:42.914942 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d4ffd6-6f03-4a9d-91f5-f6983b82587f admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:11:42.917603 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:11:42.948461 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1225/2458] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 12:11:42 2022] PUT /identity/v3/projects/bc8a1a7d21b245278a7d663ac4d78129/users/586dd2a7012d4ad1841e837eeecfd247/roles/de2d2bd26fff4748a79990f97a107164 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:11:42.949597 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c97a578-a456-4d88-9f85-177df4949275 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:11:42.949889 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c97a578-a456-4d88-9f85-177df4949275 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:11:42.950170 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c97a578-a456-4d88-9f85-177df4949275 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:11:42.980067 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-8c97a578-a456-4d88-9f85-177df4949275 None None] MFA Rules not processed for user `586dd2a7012d4ad1841e837eeecfd247`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:11:42.981267 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-8c97a578-a456-4d88-9f85-177df4949275 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:11:43.033338 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:11:43.055404 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1234/2459] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:11:42 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:11:43.069810 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d4ffd6-6f03-4a9d-91f5-f6983b82587f admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:11:43.071623 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-38d4ffd6-6f03-4a9d-91f5-f6983b82587f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:11:43.143148 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d4ffd6-6f03-4a9d-91f5-f6983b82587f admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:11:43.147892 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e123afa0-ef73-41fe-8529-3d779b812adf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b1b651c73664d08accc6edf2f79bace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:11:43.148505 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e123afa0-ef73-41fe-8529-3d779b812adf service cinder] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:11:43.148811 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e123afa0-ef73-41fe-8529-3d779b812adf service cinder] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:11:43.149087 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e123afa0-ef73-41fe-8529-3d779b812adf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:11:43.157915 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e123afa0-ef73-41fe-8529-3d779b812adf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:11:43.159484 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e123afa0-ef73-41fe-8529-3d779b812adf service cinder] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:11:43.168324 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1226/2460] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:11:43 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:11:53.052047 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=5 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:11:53.164369 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:INSERT=1,keystone:SELECT=28 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:12:07.149792 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c97a578-a456-4d88-9f85-177df4949275 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:12:07.151850 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-8c97a578-a456-4d88-9f85-177df4949275 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:12:07.230176 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c97a578-a456-4d88-9f85-177df4949275 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:12:07.235834 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a6b4fb-0d20-4e1f-9a09-9336095e20cf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bad5e1cb306b46818096940acc3810b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:12:07.236454 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a6b4fb-0d20-4e1f-9a09-9336095e20cf service nova] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:12:07.236721 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a6b4fb-0d20-4e1f-9a09-9336095e20cf service nova] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:12:07.236980 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a6b4fb-0d20-4e1f-9a09-9336095e20cf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:12:07.239136 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-a0a6b4fb-0d20-4e1f-9a09-9336095e20cf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:12:07.252541 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a6b4fb-0d20-4e1f-9a09-9336095e20cf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:12:07.254235 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a6b4fb-0d20-4e1f-9a09-9336095e20cf service nova] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:12:07.265402 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1235/2461] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:12:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:12:11.198473 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddf1dcc-1c83-4035-a96e-cd32e3f6e40d None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:12:11.199083 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddf1dcc-1c83-4035-a96e-cd32e3f6e40d None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:12:11.199248 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddf1dcc-1c83-4035-a96e-cd32e3f6e40d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:12:11.222206 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-8ddf1dcc-1c83-4035-a96e-cd32e3f6e40d None None] MFA Rules not processed for user `abfa19ab0a624de6ab082e8b1e57a220`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:12:11.223654 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-8ddf1dcc-1c83-4035-a96e-cd32e3f6e40d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:12:11.294803 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1227/2462] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:12:11 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:12:17.260797 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=34 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:12:21.140815 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a6b4fb-0d20-4e1f-9a09-9336095e20cf service nova] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:12:21.142405 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-a0a6b4fb-0d20-4e1f-9a09-9336095e20cf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:12:21.204911 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a6b4fb-0d20-4e1f-9a09-9336095e20cf service nova] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:12:21.209175 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa9a19b-8857-46ed-b787-9aa756841762 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e563fe174db4cc19ae6bf8162cc3d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:12:21.209929 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa9a19b-8857-46ed-b787-9aa756841762 service neutron] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:12:21.210197 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa9a19b-8857-46ed-b787-9aa756841762 service neutron] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:12:21.210466 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa9a19b-8857-46ed-b787-9aa756841762 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:12:21.212535 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-eaa9a19b-8857-46ed-b787-9aa756841762 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:12:21.281149 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa9a19b-8857-46ed-b787-9aa756841762 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:12:21.282526 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa9a19b-8857-46ed-b787-9aa756841762 service neutron] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:12:21.287098 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=25 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:12:21.294371 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1236/2463] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:12:21 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:12:31.288099 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=50 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:12:33.734602 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddf1dcc-1c83-4035-a96e-cd32e3f6e40d None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:12:33.803522 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddf1dcc-1c83-4035-a96e-cd32e3f6e40d None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:12:33.808553 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775f0f25-458d-400b-95b7-11a62645d426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:12:33.809188 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775f0f25-458d-400b-95b7-11a62645d426 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:12:33.809468 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775f0f25-458d-400b-95b7-11a62645d426 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:12:33.809742 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775f0f25-458d-400b-95b7-11a62645d426 admin admin] PATH_INFO: `/v3/users/586dd2a7012d4ad1841e837eeecfd247` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:12:33.810960 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775f0f25-458d-400b-95b7-11a62645d426 admin admin] RBAC: Authorizing `identity:delete_user(user_id=586dd2a7012d4ad1841e837eeecfd247)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:12:33.812883 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775f0f25-458d-400b-95b7-11a62645d426 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:12:33.861973 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-775f0f25-458d-400b-95b7-11a62645d426 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '586dd2a7012d4ad1841e837eeecfd247'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:12:33.874842 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-775f0f25-458d-400b-95b7-11a62645d426 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '586dd2a7012d4ad1841e837eeecfd247'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:12:33.884575 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-775f0f25-458d-400b-95b7-11a62645d426 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '586dd2a7012d4ad1841e837eeecfd247'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:12:33.885327 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-775f0f25-458d-400b-95b7-11a62645d426 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '586dd2a7012d4ad1841e837eeecfd247'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:12:33.892865 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-775f0f25-458d-400b-95b7-11a62645d426 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '586dd2a7012d4ad1841e837eeecfd247'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:12:33.899454 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1228/2464] 10.222.0.59 () {58 vars in 1224 bytes} [Tue Oct 11 12:12:33 2022] DELETE /identity/v3/users/586dd2a7012d4ad1841e837eeecfd247 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:12:33.904019 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0aa943-8664-4742-ab71-d9319b57cbf3 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:12:33.904499 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0aa943-8664-4742-ab71-d9319b57cbf3 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:12:33.904620 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0aa943-8664-4742-ab71-d9319b57cbf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:12:33.924350 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-2c0aa943-8664-4742-ab71-d9319b57cbf3 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:12:33.925562 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-2c0aa943-8664-4742-ab71-d9319b57cbf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:12:33.988613 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1237/2465] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:12:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:12:33.999444 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775f0f25-458d-400b-95b7-11a62645d426 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:12:34.001597 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-775f0f25-458d-400b-95b7-11a62645d426 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:12:34.068623 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775f0f25-458d-400b-95b7-11a62645d426 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:12:34.075485 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b765812-0478-478d-9b26-e3a162f08071 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e563fe174db4cc19ae6bf8162cc3d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:12:34.076315 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b765812-0478-478d-9b26-e3a162f08071 service neutron] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:12:34.076709 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b765812-0478-478d-9b26-e3a162f08071 service neutron] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:12:34.077000 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b765812-0478-478d-9b26-e3a162f08071 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:12:34.087659 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b765812-0478-478d-9b26-e3a162f08071 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:12:34.089135 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b765812-0478-478d-9b26-e3a162f08071 service neutron] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:12:34.098857 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1229/2466] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:12:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:12:34.367879 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0aa943-8664-4742-ab71-d9319b57cbf3 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:12:34.369607 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-2c0aa943-8664-4742-ab71-d9319b57cbf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:12:34.382012 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0aa943-8664-4742-ab71-d9319b57cbf3 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:12:34.387070 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec52bc4e-36dc-488a-ac9f-69f87d7245f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:12:34.387831 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec52bc4e-36dc-488a-ac9f-69f87d7245f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:12:34.388201 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec52bc4e-36dc-488a-ac9f-69f87d7245f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:12:34.388566 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec52bc4e-36dc-488a-ac9f-69f87d7245f9 admin admin] PATH_INFO: `/v3/projects/bc8a1a7d21b245278a7d663ac4d78129` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:12:34.389755 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec52bc4e-36dc-488a-ac9f-69f87d7245f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc8a1a7d21b245278a7d663ac4d78129)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:12:34.391911 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec52bc4e-36dc-488a-ac9f-69f87d7245f9 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:12:34.447215 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-ec52bc4e-36dc-488a-ac9f-69f87d7245f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc8a1a7d21b245278a7d663ac4d78129', 'request_id': 'req-ec52bc4e-36dc-488a-ac9f-69f87d7245f9'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:12:34.453647 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-ec52bc4e-36dc-488a-ac9f-69f87d7245f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc8a1a7d21b245278a7d663ac4d78129', 'request_id': 'req-ec52bc4e-36dc-488a-ac9f-69f87d7245f9'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:12:34.460334 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-ec52bc4e-36dc-488a-ac9f-69f87d7245f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc8a1a7d21b245278a7d663ac4d78129', 'request_id': 'req-ec52bc4e-36dc-488a-ac9f-69f87d7245f9'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:12:34.475525 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-ec52bc4e-36dc-488a-ac9f-69f87d7245f9 admin admin] The token cache is being invalidate because project bc8a1a7d21b245278a7d663ac4d78129 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81432) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 12:12:34.475872 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-ec52bc4e-36dc-488a-ac9f-69f87d7245f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:12:34.476556 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-ec52bc4e-36dc-488a-ac9f-69f87d7245f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:12:34.478171 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1238/2467] 10.222.0.59 () {58 vars in 1233 bytes} [Tue Oct 11 12:12:34 2022] DELETE /identity/v3/projects/bc8a1a7d21b245278a7d663ac4d78129 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:12:34.483315 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bbcc10-06b7-4e60-ac46-d2ab03450d21 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:12:34.483551 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bbcc10-06b7-4e60-ac46-d2ab03450d21 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:12:34.483830 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bbcc10-06b7-4e60-ac46-d2ab03450d21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:12:34.504692 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-b5bbcc10-06b7-4e60-ac46-d2ab03450d21 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:12:34.505949 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-b5bbcc10-06b7-4e60-ac46-d2ab03450d21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:12:34.524874 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:12:34.570855 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1230/2468] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:12:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:12:34.581765 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eac9846-94a8-4788-8bd2-45a22f1e822b None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:12:34.582082 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eac9846-94a8-4788-8bd2-45a22f1e822b None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:12:34.582347 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eac9846-94a8-4788-8bd2-45a22f1e822b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:12:34.604160 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-9eac9846-94a8-4788-8bd2-45a22f1e822b None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:12:34.605340 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-9eac9846-94a8-4788-8bd2-45a22f1e822b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:12:34.615213 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1239/2469] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:12:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:12:34.620363 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bbcc10-06b7-4e60-ac46-d2ab03450d21 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:12:34.635720 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bbcc10-06b7-4e60-ac46-d2ab03450d21 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:12:34.644496 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8692a899-c97d-4eb8-b8b4-e4b01beba0ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:12:34.645311 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8692a899-c97d-4eb8-b8b4-e4b01beba0ff admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:12:34.645582 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8692a899-c97d-4eb8-b8b4-e4b01beba0ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:12:34.646066 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8692a899-c97d-4eb8-b8b4-e4b01beba0ff admin admin] PATH_INFO: `/v3/domains` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:12:34.647017 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8692a899-c97d-4eb8-b8b4-e4b01beba0ff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 12:12:34.647450 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8692a899-c97d-4eb8-b8b4-e4b01beba0ff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:12:34.649918 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8692a899-c97d-4eb8-b8b4-e4b01beba0ff admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:12:34.668290 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1231/2470] 10.222.0.59 () {58 vars in 1153 bytes} [Tue Oct 11 12:12:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 12:12:34.674300 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eac9846-94a8-4788-8bd2-45a22f1e822b None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:12:34.685973 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eac9846-94a8-4788-8bd2-45a22f1e822b None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:12:34.692922 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5feae801-ea51-46e3-9425-2c9fd41cd077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:12:34.693738 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feae801-ea51-46e3-9425-2c9fd41cd077 admin admin] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:12:34.694090 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feae801-ea51-46e3-9425-2c9fd41cd077 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:12:34.694463 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feae801-ea51-46e3-9425-2c9fd41cd077 admin admin] PATH_INFO: `/v3/projects` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:12:34.695440 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5feae801-ea51-46e3-9425-2c9fd41cd077 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:12:34.698278 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5feae801-ea51-46e3-9425-2c9fd41cd077 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:12:34.730578 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1240/2471] 10.222.0.59 () {60 vars in 1153 bytes} [Tue Oct 11 12:12:34 2022] POST /identity/v3/projects => generated 399 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 12:12:34.737013 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8692a899-c97d-4eb8-b8b4-e4b01beba0ff admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:12:34.749650 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8692a899-c97d-4eb8-b8b4-e4b01beba0ff admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:12:34.822055 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d745e7-32a2-4152-b6d9-bf454d98bda8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:12:34.822708 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d745e7-32a2-4152-b6d9-bf454d98bda8 admin admin] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:12:34.822990 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d745e7-32a2-4152-b6d9-bf454d98bda8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:12:34.823261 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d745e7-32a2-4152-b6d9-bf454d98bda8 admin admin] PATH_INFO: `/v3/users` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:12:34.823883 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d745e7-32a2-4152-b6d9-bf454d98bda8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:12:34.825797 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d745e7-32a2-4152-b6d9-bf454d98bda8 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:12:34.849799 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1232/2472] 10.222.0.59 () {60 vars in 1144 bytes} [Tue Oct 11 12:12:34 2022] POST /identity/v3/users => generated 300 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:12:34.854243 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5feae801-ea51-46e3-9425-2c9fd41cd077 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:12:34.864750 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5feae801-ea51-46e3-9425-2c9fd41cd077 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:12:34.869942 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1183d9fe-48e5-4b21-9d77-dca9c367d860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:12:34.870539 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1183d9fe-48e5-4b21-9d77-dca9c367d860 admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:12:34.870861 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1183d9fe-48e5-4b21-9d77-dca9c367d860 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:12:34.871134 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1183d9fe-48e5-4b21-9d77-dca9c367d860 admin admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:12:34.871867 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1183d9fe-48e5-4b21-9d77-dca9c367d860 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:12:34.873469 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1183d9fe-48e5-4b21-9d77-dca9c367d860 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:12:34.884283 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1241/2473] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:12:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:12:34.888436 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d745e7-32a2-4152-b6d9-bf454d98bda8 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:12:34.899553 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d745e7-32a2-4152-b6d9-bf454d98bda8 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:12:34.905497 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa1dcca-3203-40e1-bd2e-9680ce3f7977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:12:34.906108 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa1dcca-3203-40e1-bd2e-9680ce3f7977 admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:12:34.906386 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa1dcca-3203-40e1-bd2e-9680ce3f7977 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:12:34.906793 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa1dcca-3203-40e1-bd2e-9680ce3f7977 admin admin] PATH_INFO: `/v3/roles` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:12:34.907537 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa1dcca-3203-40e1-bd2e-9680ce3f7977 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:12:34.909163 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa1dcca-3203-40e1-bd2e-9680ce3f7977 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:12:34.920899 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1233/2474] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:12:34 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:12:34.925511 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1183d9fe-48e5-4b21-9d77-dca9c367d860 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:12:34.936391 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1183d9fe-48e5-4b21-9d77-dca9c367d860 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:12:34.941962 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e93d9a-83b2-429b-8c0e-c5426cfa4e16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:12:34.942629 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e93d9a-83b2-429b-8c0e-c5426cfa4e16 admin admin] REQUEST_METHOD: `PUT` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:12:34.942988 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e93d9a-83b2-429b-8c0e-c5426cfa4e16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:12:34.943202 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e93d9a-83b2-429b-8c0e-c5426cfa4e16 admin admin] PATH_INFO: `/v3/projects/72ce90486fdd4574a981df99e1ffdc9e/users/a41cebb525204ace84ebc1161f1ee0a1/roles/de2d2bd26fff4748a79990f97a107164` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:12:34.961647 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e93d9a-83b2-429b-8c0e-c5426cfa4e16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=72ce90486fdd4574a981df99e1ffdc9e, user_id=a41cebb525204ace84ebc1161f1ee0a1, role_id=de2d2bd26fff4748a79990f97a107164)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:12:34.964521 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e93d9a-83b2-429b-8c0e-c5426cfa4e16 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:12:34.981538 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1242/2475] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 12:12:34 2022] PUT /identity/v3/projects/72ce90486fdd4574a981df99e1ffdc9e/users/a41cebb525204ace84ebc1161f1ee0a1/roles/de2d2bd26fff4748a79990f97a107164 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:12:35.055254 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27565d7-9f5f-4035-8da9-b8b141e138b8 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:12:35.055544 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27565d7-9f5f-4035-8da9-b8b141e138b8 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:12:35.055800 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27565d7-9f5f-4035-8da9-b8b141e138b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:12:35.091502 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-b27565d7-9f5f-4035-8da9-b8b141e138b8 None None] MFA Rules not processed for user `a41cebb525204ace84ebc1161f1ee0a1`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:12:35.092753 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-b27565d7-9f5f-4035-8da9-b8b141e138b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:12:35.160326 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1234/2476] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:12:35 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:12:35.172046 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e93d9a-83b2-429b-8c0e-c5426cfa4e16 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:12:35.174046 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-67e93d9a-83b2-429b-8c0e-c5426cfa4e16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:12:35.196261 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:12:35.238259 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e93d9a-83b2-429b-8c0e-c5426cfa4e16 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:12:35.242140 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ba3e76-d296-40f3-a361-a98f8126f8b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b1b651c73664d08accc6edf2f79bace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:12:35.242846 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ba3e76-d296-40f3-a361-a98f8126f8b1 service cinder] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:12:35.243185 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ba3e76-d296-40f3-a361-a98f8126f8b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:12:35.243424 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ba3e76-d296-40f3-a361-a98f8126f8b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:12:35.251984 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ba3e76-d296-40f3-a361-a98f8126f8b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:12:35.254800 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ba3e76-d296-40f3-a361-a98f8126f8b1 service cinder] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:12:35.262810 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1243/2477] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:12:35 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:12:44.446795 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27565d7-9f5f-4035-8da9-b8b141e138b8 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:12:44.467078 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:12:44.515272 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27565d7-9f5f-4035-8da9-b8b141e138b8 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:12:44.521793 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576e17cc-77b4-4374-9c02-eae1946e537a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:12:44.522438 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576e17cc-77b4-4374-9c02-eae1946e537a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:12:44.522755 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576e17cc-77b4-4374-9c02-eae1946e537a admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:12:44.523030 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576e17cc-77b4-4374-9c02-eae1946e537a admin admin] PATH_INFO: `/v3/users/a41cebb525204ace84ebc1161f1ee0a1` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:12:44.524279 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576e17cc-77b4-4374-9c02-eae1946e537a admin admin] RBAC: Authorizing `identity:delete_user(user_id=a41cebb525204ace84ebc1161f1ee0a1)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:12:44.526263 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576e17cc-77b4-4374-9c02-eae1946e537a admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:12:44.575697 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-576e17cc-77b4-4374-9c02-eae1946e537a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a41cebb525204ace84ebc1161f1ee0a1'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:12:44.587686 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-576e17cc-77b4-4374-9c02-eae1946e537a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a41cebb525204ace84ebc1161f1ee0a1'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:12:44.595561 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-576e17cc-77b4-4374-9c02-eae1946e537a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a41cebb525204ace84ebc1161f1ee0a1'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:12:44.596242 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-576e17cc-77b4-4374-9c02-eae1946e537a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a41cebb525204ace84ebc1161f1ee0a1'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:12:44.603333 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-576e17cc-77b4-4374-9c02-eae1946e537a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a41cebb525204ace84ebc1161f1ee0a1'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:12:44.609107 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1235/2478] 10.222.0.59 () {58 vars in 1224 bytes} [Tue Oct 11 12:12:44 2022] DELETE /identity/v3/users/a41cebb525204ace84ebc1161f1ee0a1 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:12:44.613946 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c311c7-ca92-4bf1-b78a-8febf268a7a3 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:12:44.614497 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c311c7-ca92-4bf1-b78a-8febf268a7a3 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:12:44.614497 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c311c7-ca92-4bf1-b78a-8febf268a7a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:12:44.634875 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-e6c311c7-ca92-4bf1-b78a-8febf268a7a3 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:12:44.635980 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-e6c311c7-ca92-4bf1-b78a-8febf268a7a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:12:44.699330 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1244/2479] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:12:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:12:44.709786 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576e17cc-77b4-4374-9c02-eae1946e537a admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:12:44.711258 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-576e17cc-77b4-4374-9c02-eae1946e537a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:12:44.774092 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576e17cc-77b4-4374-9c02-eae1946e537a admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:12:44.778010 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378e3b0d-2fc0-45e1-ab2f-2f5440334f8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e563fe174db4cc19ae6bf8162cc3d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:12:44.778624 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378e3b0d-2fc0-45e1-ab2f-2f5440334f8f service neutron] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:12:44.778923 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378e3b0d-2fc0-45e1-ab2f-2f5440334f8f service neutron] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:12:44.779199 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378e3b0d-2fc0-45e1-ab2f-2f5440334f8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:12:44.789071 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378e3b0d-2fc0-45e1-ab2f-2f5440334f8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:12:44.790496 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378e3b0d-2fc0-45e1-ab2f-2f5440334f8f service neutron] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:12:44.800269 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1236/2480] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:12:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 11 12:12:45.083872 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c311c7-ca92-4bf1-b78a-8febf268a7a3 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:12:45.085363 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-e6c311c7-ca92-4bf1-b78a-8febf268a7a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:12:45.099165 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c311c7-ca92-4bf1-b78a-8febf268a7a3 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:12:45.104097 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6213e6d5-c028-4aad-9add-508864075d28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:12:45.104734 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6213e6d5-c028-4aad-9add-508864075d28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:12:45.105031 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6213e6d5-c028-4aad-9add-508864075d28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:12:45.105303 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6213e6d5-c028-4aad-9add-508864075d28 admin admin] PATH_INFO: `/v3/projects/72ce90486fdd4574a981df99e1ffdc9e` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:12:45.106412 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6213e6d5-c028-4aad-9add-508864075d28 admin admin] RBAC: Authorizing `identity:delete_project(project_id=72ce90486fdd4574a981df99e1ffdc9e)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:12:45.108377 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6213e6d5-c028-4aad-9add-508864075d28 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:12:45.161665 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-6213e6d5-c028-4aad-9add-508864075d28 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72ce90486fdd4574a981df99e1ffdc9e', 'request_id': 'req-6213e6d5-c028-4aad-9add-508864075d28'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:12:45.167458 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-6213e6d5-c028-4aad-9add-508864075d28 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72ce90486fdd4574a981df99e1ffdc9e', 'request_id': 'req-6213e6d5-c028-4aad-9add-508864075d28'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:12:45.174156 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-6213e6d5-c028-4aad-9add-508864075d28 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72ce90486fdd4574a981df99e1ffdc9e', 'request_id': 'req-6213e6d5-c028-4aad-9add-508864075d28'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:12:45.190805 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-6213e6d5-c028-4aad-9add-508864075d28 admin admin] The token cache is being invalidate because project 72ce90486fdd4574a981df99e1ffdc9e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81432) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 12:12:45.191096 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-6213e6d5-c028-4aad-9add-508864075d28 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:12:45.191840 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-6213e6d5-c028-4aad-9add-508864075d28 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:12:45.193496 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1245/2481] 10.222.0.59 () {58 vars in 1233 bytes} [Tue Oct 11 12:12:45 2022] DELETE /identity/v3/projects/72ce90486fdd4574a981df99e1ffdc9e => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:12:45.198650 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecce1b6-30ef-4b34-9536-657964d72238 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:12:45.198934 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecce1b6-30ef-4b34-9536-657964d72238 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:12:45.199208 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecce1b6-30ef-4b34-9536-657964d72238 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:12:45.218927 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-7ecce1b6-30ef-4b34-9536-657964d72238 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:12:45.220158 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-7ecce1b6-30ef-4b34-9536-657964d72238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:12:45.255330 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:12:45.291707 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1237/2482] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:12:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:12:45.304423 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5edc56-1067-4ed9-9efa-c6fa37510ecc None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:12:45.304722 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5edc56-1067-4ed9-9efa-c6fa37510ecc None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:12:45.305062 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5edc56-1067-4ed9-9efa-c6fa37510ecc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:12:45.328303 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-df5edc56-1067-4ed9-9efa-c6fa37510ecc None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:12:45.329509 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-df5edc56-1067-4ed9-9efa-c6fa37510ecc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:12:45.338526 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1246/2483] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:12:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:12:45.343139 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecce1b6-30ef-4b34-9536-657964d72238 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:12:45.353084 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecce1b6-30ef-4b34-9536-657964d72238 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:12:45.359108 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9cbc78-0938-42a4-a595-3e6eec042553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:12:45.359715 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9cbc78-0938-42a4-a595-3e6eec042553 admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:12:45.359990 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9cbc78-0938-42a4-a595-3e6eec042553 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:12:45.360378 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9cbc78-0938-42a4-a595-3e6eec042553 admin admin] PATH_INFO: `/v3/domains` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:12:45.361074 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9cbc78-0938-42a4-a595-3e6eec042553 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 12:12:45.361367 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9cbc78-0938-42a4-a595-3e6eec042553 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:12:45.363019 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9cbc78-0938-42a4-a595-3e6eec042553 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:12:45.375570 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1238/2484] 10.222.0.59 () {58 vars in 1153 bytes} [Tue Oct 11 12:12:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 11 12:12:45.380037 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5edc56-1067-4ed9-9efa-c6fa37510ecc None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:12:45.389692 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5edc56-1067-4ed9-9efa-c6fa37510ecc None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:12:45.395605 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e8f912-7278-4355-891e-b7c8bb2d3ff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:12:45.396126 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e8f912-7278-4355-891e-b7c8bb2d3ff4 admin admin] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:12:45.396403 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e8f912-7278-4355-891e-b7c8bb2d3ff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:12:45.396679 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e8f912-7278-4355-891e-b7c8bb2d3ff4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:12:45.397328 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e8f912-7278-4355-891e-b7c8bb2d3ff4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:12:45.399346 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e8f912-7278-4355-891e-b7c8bb2d3ff4 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:12:45.429550 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1247/2485] 10.222.0.59 () {60 vars in 1153 bytes} [Tue Oct 11 12:12:45 2022] POST /identity/v3/projects => generated 377 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 12:12:45.434052 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9cbc78-0938-42a4-a595-3e6eec042553 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:12:45.444427 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9cbc78-0938-42a4-a595-3e6eec042553 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:12:45.501843 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80cf822-a9d1-4053-97ce-e87f7cedd253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:12:45.502533 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80cf822-a9d1-4053-97ce-e87f7cedd253 admin admin] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:12:45.502732 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80cf822-a9d1-4053-97ce-e87f7cedd253 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:12:45.503004 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80cf822-a9d1-4053-97ce-e87f7cedd253 admin admin] PATH_INFO: `/v3/users` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:12:45.503623 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80cf822-a9d1-4053-97ce-e87f7cedd253 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:12:45.505520 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80cf822-a9d1-4053-97ce-e87f7cedd253 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:12:45.532434 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1239/2486] 10.222.0.59 () {60 vars in 1144 bytes} [Tue Oct 11 12:12:45 2022] POST /identity/v3/users => generated 289 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:12:45.538921 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e8f912-7278-4355-891e-b7c8bb2d3ff4 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:12:45.553621 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e8f912-7278-4355-891e-b7c8bb2d3ff4 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:12:45.558690 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e42fa3-7f47-4abf-9644-b7dc3182df7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:12:45.559294 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e42fa3-7f47-4abf-9644-b7dc3182df7d admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:12:45.559575 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e42fa3-7f47-4abf-9644-b7dc3182df7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:12:45.559845 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e42fa3-7f47-4abf-9644-b7dc3182df7d admin admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:12:45.560483 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e42fa3-7f47-4abf-9644-b7dc3182df7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:12:45.562130 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e42fa3-7f47-4abf-9644-b7dc3182df7d admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:12:45.572523 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1248/2487] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:12:45 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:12:45.576849 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80cf822-a9d1-4053-97ce-e87f7cedd253 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:12:45.587747 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80cf822-a9d1-4053-97ce-e87f7cedd253 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:12:45.593049 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5653b7ae-8110-4720-93e0-164033163fa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:12:45.593641 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5653b7ae-8110-4720-93e0-164033163fa5 admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:12:45.593903 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5653b7ae-8110-4720-93e0-164033163fa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:12:45.594172 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5653b7ae-8110-4720-93e0-164033163fa5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:12:45.594841 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5653b7ae-8110-4720-93e0-164033163fa5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:12:45.596436 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5653b7ae-8110-4720-93e0-164033163fa5 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:12:45.607172 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1240/2488] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:12:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:12:45.611608 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e42fa3-7f47-4abf-9644-b7dc3182df7d admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:12:45.623106 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e42fa3-7f47-4abf-9644-b7dc3182df7d admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:12:45.629710 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b445dd07-b656-4e15-ab73-2606e3845de6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:12:45.630485 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b445dd07-b656-4e15-ab73-2606e3845de6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:12:45.630784 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b445dd07-b656-4e15-ab73-2606e3845de6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:12:45.631053 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b445dd07-b656-4e15-ab73-2606e3845de6 admin admin] PATH_INFO: `/v3/projects/4f2eaa1959394ae3a4b4b27d00b917d1/users/82c54e3aa3aa46a28373acb91f32cc9a/roles/de2d2bd26fff4748a79990f97a107164` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:12:45.633638 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:12:45.648716 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b445dd07-b656-4e15-ab73-2606e3845de6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f2eaa1959394ae3a4b4b27d00b917d1, user_id=82c54e3aa3aa46a28373acb91f32cc9a, role_id=de2d2bd26fff4748a79990f97a107164)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:12:45.651724 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b445dd07-b656-4e15-ab73-2606e3845de6 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:12:45.665771 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1249/2489] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 12:12:45 2022] PUT /identity/v3/projects/4f2eaa1959394ae3a4b4b27d00b917d1/users/82c54e3aa3aa46a28373acb91f32cc9a/roles/de2d2bd26fff4748a79990f97a107164 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:12:45.671147 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea31c51-7f7c-48d3-95e2-a55c1408ca4a None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:12:45.671429 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea31c51-7f7c-48d3-95e2-a55c1408ca4a None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:12:45.671699 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea31c51-7f7c-48d3-95e2-a55c1408ca4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:12:45.692606 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-1ea31c51-7f7c-48d3-95e2-a55c1408ca4a None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:12:45.693752 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-1ea31c51-7f7c-48d3-95e2-a55c1408ca4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:12:45.761964 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1241/2490] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:12:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:12:45.772749 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b445dd07-b656-4e15-ab73-2606e3845de6 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:12:45.775621 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-b445dd07-b656-4e15-ab73-2606e3845de6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:12:45.848321 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b445dd07-b656-4e15-ab73-2606e3845de6 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:12:45.852565 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abc8688-12a2-4568-8573-92920faaf578 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e563fe174db4cc19ae6bf8162cc3d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:12:45.853179 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abc8688-12a2-4568-8573-92920faaf578 service neutron] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:12:45.853459 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abc8688-12a2-4568-8573-92920faaf578 service neutron] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:12:45.853728 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abc8688-12a2-4568-8573-92920faaf578 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:12:45.864133 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abc8688-12a2-4568-8573-92920faaf578 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:12:45.865580 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abc8688-12a2-4568-8573-92920faaf578 service neutron] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:12:45.875272 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1250/2491] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:12:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:12:49.871915 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb05ea0d-0193-4f7d-94f5-d803177e9e14 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:12:49.872414 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb05ea0d-0193-4f7d-94f5-d803177e9e14 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:12:49.872574 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb05ea0d-0193-4f7d-94f5-d803177e9e14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:12:49.904852 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-fb05ea0d-0193-4f7d-94f5-d803177e9e14 None None] MFA Rules not processed for user `82c54e3aa3aa46a28373acb91f32cc9a`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:12:49.906304 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-fb05ea0d-0193-4f7d-94f5-d803177e9e14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:12:49.945752 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:12:49.990609 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1242/2492] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:12:49 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:12:49.997696 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abc8688-12a2-4568-8573-92920faaf578 service neutron] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:12:50.006581 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abc8688-12a2-4568-8573-92920faaf578 service neutron] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:12:50.012190 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0a37b6-d607-4b73-8984-a29e84b851a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:12:50.012851 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0a37b6-d607-4b73-8984-a29e84b851a3 admin admin] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:12:50.013133 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0a37b6-d607-4b73-8984-a29e84b851a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:12:50.013414 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0a37b6-d607-4b73-8984-a29e84b851a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:12:50.014072 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0a37b6-d607-4b73-8984-a29e84b851a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:12:50.016114 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0a37b6-d607-4b73-8984-a29e84b851a3 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:12:50.048503 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1251/2493] 10.222.0.59 () {60 vars in 1153 bytes} [Tue Oct 11 12:12:49 2022] POST /identity/v3/projects => generated 377 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:12:50.052516 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb05ea0d-0193-4f7d-94f5-d803177e9e14 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:12:50.064296 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb05ea0d-0193-4f7d-94f5-d803177e9e14 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:12:50.125662 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13780ba3-899d-4fb9-8af8-b3fbee08c046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:12:50.126284 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13780ba3-899d-4fb9-8af8-b3fbee08c046 admin admin] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:12:50.126573 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13780ba3-899d-4fb9-8af8-b3fbee08c046 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:12:50.127000 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13780ba3-899d-4fb9-8af8-b3fbee08c046 admin admin] PATH_INFO: `/v3/users` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:12:50.127690 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13780ba3-899d-4fb9-8af8-b3fbee08c046 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:12:50.129652 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13780ba3-899d-4fb9-8af8-b3fbee08c046 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:12:50.155008 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1243/2494] 10.222.0.59 () {60 vars in 1144 bytes} [Tue Oct 11 12:12:50 2022] POST /identity/v3/users => generated 295 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:12:50.159090 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0a37b6-d607-4b73-8984-a29e84b851a3 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:12:50.190257 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0a37b6-d607-4b73-8984-a29e84b851a3 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:12:50.203222 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021e3b41-5caf-470c-b56f-1b9bdb3eccdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:12:50.203917 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021e3b41-5caf-470c-b56f-1b9bdb3eccdd admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:12:50.204199 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021e3b41-5caf-470c-b56f-1b9bdb3eccdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:12:50.204478 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021e3b41-5caf-470c-b56f-1b9bdb3eccdd admin admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:12:50.205152 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021e3b41-5caf-470c-b56f-1b9bdb3eccdd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:12:50.206925 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021e3b41-5caf-470c-b56f-1b9bdb3eccdd admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:12:50.219448 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1252/2495] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:12:50 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:12:50.226458 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13780ba3-899d-4fb9-8af8-b3fbee08c046 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:12:50.238220 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13780ba3-899d-4fb9-8af8-b3fbee08c046 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:12:50.243174 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9b085d-7273-42c6-967c-e3cd20fbc2fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:12:50.243842 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9b085d-7273-42c6-967c-e3cd20fbc2fb admin admin] REQUEST_METHOD: `PUT` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:12:50.244120 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9b085d-7273-42c6-967c-e3cd20fbc2fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:12:50.244390 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9b085d-7273-42c6-967c-e3cd20fbc2fb admin admin] PATH_INFO: `/v3/projects/a0ad01844e424ab28fe744762385d05d/users/f09e4d3c403d40ffad9ca134fa802157/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:12:50.260644 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9b085d-7273-42c6-967c-e3cd20fbc2fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0ad01844e424ab28fe744762385d05d, user_id=f09e4d3c403d40ffad9ca134fa802157, role_id=5cf1f4bd2fef46ab9c6eff56d52f9f69)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:12:50.263335 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9b085d-7273-42c6-967c-e3cd20fbc2fb admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:12:50.279765 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1244/2496] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 12:12:50 2022] PUT /identity/v3/projects/a0ad01844e424ab28fe744762385d05d/users/f09e4d3c403d40ffad9ca134fa802157/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:12:50.284332 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021e3b41-5caf-470c-b56f-1b9bdb3eccdd admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:12:50.295263 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021e3b41-5caf-470c-b56f-1b9bdb3eccdd admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:12:50.357542 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a7efd8-c76d-4f62-8f72-b0ce47d503c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:12:50.358164 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a7efd8-c76d-4f62-8f72-b0ce47d503c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:12:50.358441 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a7efd8-c76d-4f62-8f72-b0ce47d503c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:12:50.358782 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a7efd8-c76d-4f62-8f72-b0ce47d503c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:12:50.359425 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a7efd8-c76d-4f62-8f72-b0ce47d503c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:12:50.361072 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a7efd8-c76d-4f62-8f72-b0ce47d503c3 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:12:50.372788 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1253/2497] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:12:50 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:12:50.377125 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9b085d-7273-42c6-967c-e3cd20fbc2fb admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:12:50.388805 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9b085d-7273-42c6-967c-e3cd20fbc2fb admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:12:50.393922 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb704026-3abf-4850-b0cc-e713014c705a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:12:50.394801 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb704026-3abf-4850-b0cc-e713014c705a admin admin] REQUEST_METHOD: `PUT` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:12:50.395138 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb704026-3abf-4850-b0cc-e713014c705a admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:12:50.395953 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb704026-3abf-4850-b0cc-e713014c705a admin admin] PATH_INFO: `/v3/projects/a0ad01844e424ab28fe744762385d05d/users/f09e4d3c403d40ffad9ca134fa802157/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:12:50.397910 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb704026-3abf-4850-b0cc-e713014c705a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0ad01844e424ab28fe744762385d05d, user_id=f09e4d3c403d40ffad9ca134fa802157, role_id=5cf1f4bd2fef46ab9c6eff56d52f9f69)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:12:50.400629 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb704026-3abf-4850-b0cc-e713014c705a admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:12:50.411982 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1245/2498] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 12:12:50 2022] PUT /identity/v3/projects/a0ad01844e424ab28fe744762385d05d/users/f09e4d3c403d40ffad9ca134fa802157/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:12:54.910535 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bdb5b8-4a96-4edb-8ffe-e0a960f27c68 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:12:54.911079 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bdb5b8-4a96-4edb-8ffe-e0a960f27c68 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:12:54.911174 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bdb5b8-4a96-4edb-8ffe-e0a960f27c68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:12:54.943349 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-51bdb5b8-4a96-4edb-8ffe-e0a960f27c68 None None] MFA Rules not processed for user `f09e4d3c403d40ffad9ca134fa802157`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:12:54.945070 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-51bdb5b8-4a96-4edb-8ffe-e0a960f27c68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:12:54.997015 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:12:55.037202 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1254/2499] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:12:54 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:12:55.053242 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb704026-3abf-4850-b0cc-e713014c705a admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:12:55.056022 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-fb704026-3abf-4850-b0cc-e713014c705a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:12:55.193363 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb704026-3abf-4850-b0cc-e713014c705a admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:12:55.199779 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42312ea-0b55-4f77-9ef0-dcadf0396250 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bad5e1cb306b46818096940acc3810b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:12:55.201031 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42312ea-0b55-4f77-9ef0-dcadf0396250 service nova] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:12:55.201520 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42312ea-0b55-4f77-9ef0-dcadf0396250 service nova] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:12:55.201963 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42312ea-0b55-4f77-9ef0-dcadf0396250 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:12:55.231985 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:12:55.277570 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42312ea-0b55-4f77-9ef0-dcadf0396250 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:12:55.280238 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42312ea-0b55-4f77-9ef0-dcadf0396250 service nova] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:12:55.290368 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1246/2500] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:12:55 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:13:03.526821 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bdb5b8-4a96-4edb-8ffe-e0a960f27c68 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:13:03.585568 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bdb5b8-4a96-4edb-8ffe-e0a960f27c68 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:13:03.589374 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a352ab-d9b1-47cd-a488-a59c0c22a920 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e563fe174db4cc19ae6bf8162cc3d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:13:03.590067 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a352ab-d9b1-47cd-a488-a59c0c22a920 service neutron] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:13:03.590400 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a352ab-d9b1-47cd-a488-a59c0c22a920 service neutron] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:13:03.590648 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a352ab-d9b1-47cd-a488-a59c0c22a920 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:13:03.592476 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-b8a352ab-d9b1-47cd-a488-a59c0c22a920 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:13:03.656092 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a352ab-d9b1-47cd-a488-a59c0c22a920 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:13:03.657554 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a352ab-d9b1-47cd-a488-a59c0c22a920 service neutron] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:13:03.667594 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1255/2501] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:13:03 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:13:04.551403 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42312ea-0b55-4f77-9ef0-dcadf0396250 service nova] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:13:04.564158 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42312ea-0b55-4f77-9ef0-dcadf0396250 service nova] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:13:04.571197 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cca8c0-43db-4878-8334-cc42f32bdb2b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bad5e1cb306b46818096940acc3810b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:13:04.571832 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cca8c0-43db-4878-8334-cc42f32bdb2b service nova] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:13:04.572096 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cca8c0-43db-4878-8334-cc42f32bdb2b service nova] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:13:04.572373 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cca8c0-43db-4878-8334-cc42f32bdb2b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:13:04.574214 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-57cca8c0-43db-4878-8334-cc42f32bdb2b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:13:04.588982 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cca8c0-43db-4878-8334-cc42f32bdb2b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:13:04.590718 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cca8c0-43db-4878-8334-cc42f32bdb2b service nova] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:13:04.602484 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1247/2502] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:13:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:13:09.734490 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a352ab-d9b1-47cd-a488-a59c0c22a920 service neutron] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:13:09.735954 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-b8a352ab-d9b1-47cd-a488-a59c0c22a920 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:13:09.747543 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a352ab-d9b1-47cd-a488-a59c0c22a920 service neutron] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:13:09.751476 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a1afd4-2508-4b6e-8568-4d9a03e1d895 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e563fe174db4cc19ae6bf8162cc3d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:13:09.752104 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a1afd4-2508-4b6e-8568-4d9a03e1d895 service neutron] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:13:09.752383 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a1afd4-2508-4b6e-8568-4d9a03e1d895 service neutron] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:13:09.752653 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a1afd4-2508-4b6e-8568-4d9a03e1d895 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:13:09.754428 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-b0a1afd4-2508-4b6e-8568-4d9a03e1d895 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:13:09.820164 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a1afd4-2508-4b6e-8568-4d9a03e1d895 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:13:09.821628 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a1afd4-2508-4b6e-8568-4d9a03e1d895 service neutron] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:13:09.831754 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1256/2503] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:13:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:13:13.375491 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cca8c0-43db-4878-8334-cc42f32bdb2b service nova] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:13:13.376969 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-57cca8c0-43db-4878-8334-cc42f32bdb2b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:13:13.468299 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cca8c0-43db-4878-8334-cc42f32bdb2b service nova] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:13:13.473358 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31762472-8817-49e2-9600-49bca77502f6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f272cdd1bcc04b4e8261ed533cfd4fd7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:13:13.473999 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31762472-8817-49e2-9600-49bca77502f6 service placement] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:13:13.474277 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31762472-8817-49e2-9600-49bca77502f6 service placement] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:13:13.474547 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31762472-8817-49e2-9600-49bca77502f6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:13:13.476295 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-31762472-8817-49e2-9600-49bca77502f6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:13:13.489040 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31762472-8817-49e2-9600-49bca77502f6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:13:13.490772 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31762472-8817-49e2-9600-49bca77502f6 service placement] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:13:13.510040 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1248/2504] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:13:13 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:13:16.592459 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a1afd4-2508-4b6e-8568-4d9a03e1d895 service neutron] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:13:16.604809 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a1afd4-2508-4b6e-8568-4d9a03e1d895 service neutron] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:13:16.611068 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd397307-8a26-4f12-bc66-773a8faa49a1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f272cdd1bcc04b4e8261ed533cfd4fd7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:13:16.611068 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd397307-8a26-4f12-bc66-773a8faa49a1 service placement] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:13:16.611354 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd397307-8a26-4f12-bc66-773a8faa49a1 service placement] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:13:16.611450 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd397307-8a26-4f12-bc66-773a8faa49a1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:13:16.613143 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-cd397307-8a26-4f12-bc66-773a8faa49a1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:13:16.627937 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd397307-8a26-4f12-bc66-773a8faa49a1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:13:16.630093 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd397307-8a26-4f12-bc66-773a8faa49a1 service placement] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:13:16.642319 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1257/2505] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:13:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:13:16.651151 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31762472-8817-49e2-9600-49bca77502f6 service placement] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:13:16.652680 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-31762472-8817-49e2-9600-49bca77502f6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:13:16.670068 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31762472-8817-49e2-9600-49bca77502f6 service placement] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:13:16.674159 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4bf2af-c152-42c9-9109-0f8189dcfc53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:13:16.674835 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4bf2af-c152-42c9-9109-0f8189dcfc53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:13:16.675297 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4bf2af-c152-42c9-9109-0f8189dcfc53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:13:16.676123 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4bf2af-c152-42c9-9109-0f8189dcfc53 admin admin] PATH_INFO: `/v3/users/edb722b67a804915bf38f34b67845bc9` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:13:16.677768 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4bf2af-c152-42c9-9109-0f8189dcfc53 admin admin] RBAC: Authorizing `identity:delete_user(user_id=edb722b67a804915bf38f34b67845bc9)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:13:16.679751 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4bf2af-c152-42c9-9109-0f8189dcfc53 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:13:16.733241 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-aa4bf2af-c152-42c9-9109-0f8189dcfc53 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'edb722b67a804915bf38f34b67845bc9'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:13:16.745762 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-aa4bf2af-c152-42c9-9109-0f8189dcfc53 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'edb722b67a804915bf38f34b67845bc9'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:13:16.758397 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-aa4bf2af-c152-42c9-9109-0f8189dcfc53 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'edb722b67a804915bf38f34b67845bc9'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:13:16.759450 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-aa4bf2af-c152-42c9-9109-0f8189dcfc53 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'edb722b67a804915bf38f34b67845bc9'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:13:16.770740 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-aa4bf2af-c152-42c9-9109-0f8189dcfc53 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'edb722b67a804915bf38f34b67845bc9'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:13:16.779546 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1249/2506] 10.222.0.59 () {58 vars in 1224 bytes} [Tue Oct 11 12:13:16 2022] DELETE /identity/v3/users/edb722b67a804915bf38f34b67845bc9 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:13:16.965074 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd397307-8a26-4f12-bc66-773a8faa49a1 service placement] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:13:16.976899 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd397307-8a26-4f12-bc66-773a8faa49a1 service placement] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:13:16.984733 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:13:17.041179 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131a5436-f249-46e8-a63b-55ed6c00007a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:13:17.041901 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131a5436-f249-46e8-a63b-55ed6c00007a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:13:17.042180 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131a5436-f249-46e8-a63b-55ed6c00007a admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:13:17.042460 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131a5436-f249-46e8-a63b-55ed6c00007a admin admin] PATH_INFO: `/v3/projects/ef7f8c1858da4f73925e63ea06100eab` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:13:17.043584 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131a5436-f249-46e8-a63b-55ed6c00007a admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef7f8c1858da4f73925e63ea06100eab)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:13:17.045607 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131a5436-f249-46e8-a63b-55ed6c00007a admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:13:17.094289 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-131a5436-f249-46e8-a63b-55ed6c00007a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef7f8c1858da4f73925e63ea06100eab', 'request_id': 'req-131a5436-f249-46e8-a63b-55ed6c00007a'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:13:17.099592 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-131a5436-f249-46e8-a63b-55ed6c00007a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef7f8c1858da4f73925e63ea06100eab', 'request_id': 'req-131a5436-f249-46e8-a63b-55ed6c00007a'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:13:17.105737 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-131a5436-f249-46e8-a63b-55ed6c00007a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef7f8c1858da4f73925e63ea06100eab', 'request_id': 'req-131a5436-f249-46e8-a63b-55ed6c00007a'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:13:17.120071 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-131a5436-f249-46e8-a63b-55ed6c00007a admin admin] The token cache is being invalidate because project ef7f8c1858da4f73925e63ea06100eab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81432) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 12:13:17.120356 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-131a5436-f249-46e8-a63b-55ed6c00007a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:13:17.120971 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-131a5436-f249-46e8-a63b-55ed6c00007a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:13:17.122458 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1258/2507] 10.222.0.59 () {58 vars in 1233 bytes} [Tue Oct 11 12:13:16 2022] DELETE /identity/v3/projects/ef7f8c1858da4f73925e63ea06100eab => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:13:17.126722 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4bf2af-c152-42c9-9109-0f8189dcfc53 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:13:17.128121 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-aa4bf2af-c152-42c9-9109-0f8189dcfc53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:13:17.141076 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4bf2af-c152-42c9-9109-0f8189dcfc53 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:13:17.150037 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:13:17.201314 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712d356e-c5a0-421e-ba7b-a5b799540a9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:13:17.201978 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712d356e-c5a0-421e-ba7b-a5b799540a9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:13:17.202269 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712d356e-c5a0-421e-ba7b-a5b799540a9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:13:17.202540 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712d356e-c5a0-421e-ba7b-a5b799540a9f admin admin] PATH_INFO: `/v3/users/15bc8f7341dd46bb83584f5dc4d5d50d` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:13:17.204264 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712d356e-c5a0-421e-ba7b-a5b799540a9f admin admin] RBAC: Authorizing `identity:delete_user(user_id=15bc8f7341dd46bb83584f5dc4d5d50d)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:13:17.206974 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712d356e-c5a0-421e-ba7b-a5b799540a9f admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:13:17.261611 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-712d356e-c5a0-421e-ba7b-a5b799540a9f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15bc8f7341dd46bb83584f5dc4d5d50d'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:13:17.273853 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-712d356e-c5a0-421e-ba7b-a5b799540a9f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15bc8f7341dd46bb83584f5dc4d5d50d'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:13:17.283043 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-712d356e-c5a0-421e-ba7b-a5b799540a9f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15bc8f7341dd46bb83584f5dc4d5d50d'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:13:17.283825 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-712d356e-c5a0-421e-ba7b-a5b799540a9f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15bc8f7341dd46bb83584f5dc4d5d50d'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:13:17.291203 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-712d356e-c5a0-421e-ba7b-a5b799540a9f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15bc8f7341dd46bb83584f5dc4d5d50d'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:13:17.298024 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1250/2508] 10.222.0.59 () {58 vars in 1224 bytes} [Tue Oct 11 12:13:17 2022] DELETE /identity/v3/users/15bc8f7341dd46bb83584f5dc4d5d50d => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:13:17.480473 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131a5436-f249-46e8-a63b-55ed6c00007a admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:13:17.482070 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-131a5436-f249-46e8-a63b-55ed6c00007a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:13:17.495517 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131a5436-f249-46e8-a63b-55ed6c00007a admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:13:17.574038 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1d4deb-21e7-4481-8913-beefb954ce22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:13:17.574700 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1d4deb-21e7-4481-8913-beefb954ce22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:13:17.574985 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1d4deb-21e7-4481-8913-beefb954ce22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:13:17.575260 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1d4deb-21e7-4481-8913-beefb954ce22 admin admin] PATH_INFO: `/v3/projects/5b8a4b5ccf924d7da68aec4a0fbf1281` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:13:17.576462 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1d4deb-21e7-4481-8913-beefb954ce22 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b8a4b5ccf924d7da68aec4a0fbf1281)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:13:17.578396 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1d4deb-21e7-4481-8913-beefb954ce22 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:13:17.620393 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:13:17.627105 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-0d1d4deb-21e7-4481-8913-beefb954ce22 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b8a4b5ccf924d7da68aec4a0fbf1281', 'request_id': 'req-0d1d4deb-21e7-4481-8913-beefb954ce22'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:13:17.633678 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-0d1d4deb-21e7-4481-8913-beefb954ce22 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b8a4b5ccf924d7da68aec4a0fbf1281', 'request_id': 'req-0d1d4deb-21e7-4481-8913-beefb954ce22'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:13:17.640199 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-0d1d4deb-21e7-4481-8913-beefb954ce22 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b8a4b5ccf924d7da68aec4a0fbf1281', 'request_id': 'req-0d1d4deb-21e7-4481-8913-beefb954ce22'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:13:17.654340 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-0d1d4deb-21e7-4481-8913-beefb954ce22 admin admin] The token cache is being invalidate because project 5b8a4b5ccf924d7da68aec4a0fbf1281 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81432) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 12:13:17.654694 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-0d1d4deb-21e7-4481-8913-beefb954ce22 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:13:17.655399 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-0d1d4deb-21e7-4481-8913-beefb954ce22 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:13:17.656961 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1259/2509] 10.222.0.59 () {58 vars in 1233 bytes} [Tue Oct 11 12:13:17 2022] DELETE /identity/v3/projects/5b8a4b5ccf924d7da68aec4a0fbf1281 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:13:21.304004 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d31b10e-833e-4262-bc8b-545c6395a4b9 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:13:21.304560 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d31b10e-833e-4262-bc8b-545c6395a4b9 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:13:21.304560 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d31b10e-833e-4262-bc8b-545c6395a4b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:13:21.327429 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-1d31b10e-833e-4262-bc8b-545c6395a4b9 None None] MFA Rules not processed for user `82c54e3aa3aa46a28373acb91f32cc9a`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:13:21.331260 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-1d31b10e-833e-4262-bc8b-545c6395a4b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:13:21.397523 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1251/2510] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:13:21 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:13:21.738225 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1d4deb-21e7-4481-8913-beefb954ce22 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:13:21.739912 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-0d1d4deb-21e7-4481-8913-beefb954ce22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:13:21.801362 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1d4deb-21e7-4481-8913-beefb954ce22 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:13:21.805412 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db008e5-abbe-442e-bc21-1ceeadeec914 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e563fe174db4cc19ae6bf8162cc3d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:13:21.806029 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db008e5-abbe-442e-bc21-1ceeadeec914 service neutron] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:13:21.806310 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db008e5-abbe-442e-bc21-1ceeadeec914 service neutron] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:13:21.806585 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db008e5-abbe-442e-bc21-1ceeadeec914 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:13:21.808508 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-0db008e5-abbe-442e-bc21-1ceeadeec914 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:13:21.818991 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db008e5-abbe-442e-bc21-1ceeadeec914 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:13:21.820460 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db008e5-abbe-442e-bc21-1ceeadeec914 service neutron] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:13:21.828994 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1260/2511] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:13:21 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:13:22.826239 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d31b10e-833e-4262-bc8b-545c6395a4b9 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:13:22.827913 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-1d31b10e-833e-4262-bc8b-545c6395a4b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:13:22.874355 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:13:22.906306 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d31b10e-833e-4262-bc8b-545c6395a4b9 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:13:22.911987 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ffd638-9d66-4c7b-931f-7c532c549465 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f272cdd1bcc04b4e8261ed533cfd4fd7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:13:22.912605 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ffd638-9d66-4c7b-931f-7c532c549465 service placement] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:13:22.912888 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ffd638-9d66-4c7b-931f-7c532c549465 service placement] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:13:22.913161 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ffd638-9d66-4c7b-931f-7c532c549465 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:13:22.915032 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-e4ffd638-9d66-4c7b-931f-7c532c549465 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:13:22.927492 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ffd638-9d66-4c7b-931f-7c532c549465 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:13:22.929185 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ffd638-9d66-4c7b-931f-7c532c549465 service placement] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:13:22.939893 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1252/2512] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:13:22 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:13:31.825018 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=35,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:13:32.935598 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=15 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:13:40.503310 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db008e5-abbe-442e-bc21-1ceeadeec914 service neutron] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:13:40.513157 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db008e5-abbe-442e-bc21-1ceeadeec914 service neutron] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:13:40.517902 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41326ecc-5e36-4d9d-9f5f-9256135df5bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e563fe174db4cc19ae6bf8162cc3d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:13:40.518515 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41326ecc-5e36-4d9d-9f5f-9256135df5bb service neutron] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:13:40.518846 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41326ecc-5e36-4d9d-9f5f-9256135df5bb service neutron] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:13:40.519446 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41326ecc-5e36-4d9d-9f5f-9256135df5bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:13:40.521452 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-41326ecc-5e36-4d9d-9f5f-9256135df5bb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:13:40.588172 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41326ecc-5e36-4d9d-9f5f-9256135df5bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:13:40.589638 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41326ecc-5e36-4d9d-9f5f-9256135df5bb service neutron] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:13:40.599585 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1261/2513] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:13:40 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:13:50.594965 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=30 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:13:54.139434 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ffd638-9d66-4c7b-931f-7c532c549465 service placement] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:13:54.141228 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-e4ffd638-9d66-4c7b-931f-7c532c549465 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:13:54.221387 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ffd638-9d66-4c7b-931f-7c532c549465 service placement] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:13:54.227958 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1360ab05-e5fe-4382-8d04-b89f8911916d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bad5e1cb306b46818096940acc3810b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:13:54.228578 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1360ab05-e5fe-4382-8d04-b89f8911916d service nova] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:13:54.228859 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1360ab05-e5fe-4382-8d04-b89f8911916d service nova] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:13:54.229132 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1360ab05-e5fe-4382-8d04-b89f8911916d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:13:54.231409 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-1360ab05-e5fe-4382-8d04-b89f8911916d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:13:54.300763 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1360ab05-e5fe-4382-8d04-b89f8911916d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:13:54.303182 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1360ab05-e5fe-4382-8d04-b89f8911916d service nova] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:13:54.315723 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1253/2514] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:13:54 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:14:04.310943 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=56 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:14:14.113687 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0d3d22-4534-4d48-bd54-961c5ed36261 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:14:14.114234 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0d3d22-4534-4d48-bd54-961c5ed36261 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:14:14.114338 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0d3d22-4534-4d48-bd54-961c5ed36261 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:14:14.136582 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-5c0d3d22-4534-4d48-bd54-961c5ed36261 None None] MFA Rules not processed for user `8a19eceb62ce4157a4c5fe1866a89df9`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:14:14.137811 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-5c0d3d22-4534-4d48-bd54-961c5ed36261 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:14:14.198915 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1262/2515] 10.222.0.59 () {58 vars in 1009 bytes} [Tue Oct 11 12:14:14 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:14:14.247814 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1360ab05-e5fe-4382-8d04-b89f8911916d service nova] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:14:14.250060 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-1360ab05-e5fe-4382-8d04-b89f8911916d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:14:14.323348 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1360ab05-e5fe-4382-8d04-b89f8911916d service nova] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:14:14.329960 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e70b7b-3ec8-4cf8-8f3f-43d66abc192d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76e7fee1993a40d5ba4295e7daeb3e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:14:14.330588 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e70b7b-3ec8-4cf8-8f3f-43d66abc192d service swift] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:14:14.330863 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e70b7b-3ec8-4cf8-8f3f-43d66abc192d service swift] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:14:14.331132 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e70b7b-3ec8-4cf8-8f3f-43d66abc192d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:14:14.341079 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e70b7b-3ec8-4cf8-8f3f-43d66abc192d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:14:14.342687 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e70b7b-3ec8-4cf8-8f3f-43d66abc192d service swift] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:14:14.351127 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1254/2516] 10.222.0.59 () {60 vars in 1384 bytes} [Tue Oct 11 12:14:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 11 12:14:24.190374 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=25 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:14:24.347817 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=32 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:14:29.974040 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8ce668-5870-4b55-bcf7-669a1d18bf7d None None] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:14:29.974582 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8ce668-5870-4b55-bcf7-669a1d18bf7d None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:14:29.974582 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8ce668-5870-4b55-bcf7-669a1d18bf7d None None] PATH_INFO: `/` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:14:29.975960 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1263/2517] 10.222.0.59 () {54 vars in 921 bytes} [Tue Oct 11 12:14:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 11 12:14:29.981229 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e091d08f-ca92-4c6d-84cc-ddbf9122575e None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:14:29.981737 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e091d08f-ca92-4c6d-84cc-ddbf9122575e None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:14:29.981832 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e091d08f-ca92-4c6d-84cc-ddbf9122575e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:14:30.003005 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-e091d08f-ca92-4c6d-84cc-ddbf9122575e None None] MFA Rules not processed for user `bad5e1cb306b46818096940acc3810b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:14:30.004236 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-e091d08f-ca92-4c6d-84cc-ddbf9122575e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:14:30.014371 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1255/2518] 10.222.0.59 () {58 vars in 1018 bytes} [Tue Oct 11 12:14:29 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:14:30.027529 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8ce668-5870-4b55-bcf7-669a1d18bf7d None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:14:30.029477 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-ec8ce668-5870-4b55-bcf7-669a1d18bf7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:14:30.115777 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8ce668-5870-4b55-bcf7-669a1d18bf7d None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:14:30.123222 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ccaa2c-61a1-4986-a946-e8b4ef73e813 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91a00658b79a42bd91a22a2508b5de34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:14:30.124126 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ccaa2c-61a1-4986-a946-e8b4ef73e813 service glance] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:14:30.124507 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ccaa2c-61a1-4986-a946-e8b4ef73e813 service glance] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:14:30.125061 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ccaa2c-61a1-4986-a946-e8b4ef73e813 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:14:30.137491 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ccaa2c-61a1-4986-a946-e8b4ef73e813 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:14:30.138966 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ccaa2c-61a1-4986-a946-e8b4ef73e813 service glance] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:14:30.150358 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1264/2519] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:14:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:14:35.133014 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9c60b0-633a-4f77-bb98-60494cc44c7d None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:14:35.133540 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9c60b0-633a-4f77-bb98-60494cc44c7d None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:14:35.133540 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9c60b0-633a-4f77-bb98-60494cc44c7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:14:35.157977 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-dc9c60b0-633a-4f77-bb98-60494cc44c7d None None] MFA Rules not processed for user `8a19eceb62ce4157a4c5fe1866a89df9`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:14:35.159624 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-dc9c60b0-633a-4f77-bb98-60494cc44c7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:14:35.167116 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1256/2520] 10.222.0.59 () {58 vars in 1009 bytes} [Tue Oct 11 12:14:35 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:14:35.180254 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ccaa2c-61a1-4986-a946-e8b4ef73e813 service glance] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:14:35.196922 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ccaa2c-61a1-4986-a946-e8b4ef73e813 service glance] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:14:35.205673 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900d171f-de1d-4c20-b197-42c64366bd68 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76e7fee1993a40d5ba4295e7daeb3e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:14:35.206684 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900d171f-de1d-4c20-b197-42c64366bd68 service swift] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:14:35.206826 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900d171f-de1d-4c20-b197-42c64366bd68 service swift] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:14:35.207179 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900d171f-de1d-4c20-b197-42c64366bd68 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:14:35.217166 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900d171f-de1d-4c20-b197-42c64366bd68 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:14:35.218805 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900d171f-de1d-4c20-b197-42c64366bd68 service swift] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:14:35.227543 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1265/2521] 10.222.0.59 () {60 vars in 1384 bytes} [Tue Oct 11 12:14:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 11 12:14:45.152197 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=6 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:14:45.223855 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=41 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:14:55.736282 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9c60b0-633a-4f77-bb98-60494cc44c7d None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:14:55.737925 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-dc9c60b0-633a-4f77-bb98-60494cc44c7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:14:55.750114 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9c60b0-633a-4f77-bb98-60494cc44c7d None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:14:55.754250 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c051f9f8-85ad-43e7-8167-e9318ab67d48 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e563fe174db4cc19ae6bf8162cc3d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:14:55.754920 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c051f9f8-85ad-43e7-8167-e9318ab67d48 service neutron] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:14:55.755199 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c051f9f8-85ad-43e7-8167-e9318ab67d48 service neutron] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:14:55.755483 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c051f9f8-85ad-43e7-8167-e9318ab67d48 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:14:55.757375 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-c051f9f8-85ad-43e7-8167-e9318ab67d48 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:14:55.773578 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c051f9f8-85ad-43e7-8167-e9318ab67d48 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:14:55.775094 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c051f9f8-85ad-43e7-8167-e9318ab67d48 service neutron] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:14:55.786679 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1257/2522] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:14:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:14:57.667792 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900d171f-de1d-4c20-b197-42c64366bd68 service swift] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:14:57.677580 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900d171f-de1d-4c20-b197-42c64366bd68 service swift] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:14:57.681458 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a186316-b034-4cc3-ac72-53a877718f68 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e563fe174db4cc19ae6bf8162cc3d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:14:57.682207 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a186316-b034-4cc3-ac72-53a877718f68 service neutron] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:14:57.682572 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a186316-b034-4cc3-ac72-53a877718f68 service neutron] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:14:57.682911 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a186316-b034-4cc3-ac72-53a877718f68 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:14:57.684767 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-5a186316-b034-4cc3-ac72-53a877718f68 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:14:57.698902 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a186316-b034-4cc3-ac72-53a877718f68 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:14:57.700362 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a186316-b034-4cc3-ac72-53a877718f68 service neutron] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:14:57.718957 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1266/2523] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:14:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:14:58.045866 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ac46d6-8045-4f20-ae9a-7c2975e3f82c None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:14:58.046378 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ac46d6-8045-4f20-ae9a-7c2975e3f82c None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:14:58.046475 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ac46d6-8045-4f20-ae9a-7c2975e3f82c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:14:58.078096 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-00ac46d6-8045-4f20-ae9a-7c2975e3f82c None None] MFA Rules not processed for user `8a19eceb62ce4157a4c5fe1866a89df9`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:14:58.079801 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-00ac46d6-8045-4f20-ae9a-7c2975e3f82c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:14:58.088316 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1258/2524] 10.222.0.59 () {58 vars in 967 bytes} [Tue Oct 11 12:14:58 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:14:58.104952 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a186316-b034-4cc3-ac72-53a877718f68 service neutron] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:14:58.122515 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a186316-b034-4cc3-ac72-53a877718f68 service neutron] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:14:58.130191 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355c4d36-6aea-4564-9879-6e0c1071a818 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76e7fee1993a40d5ba4295e7daeb3e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:14:58.130860 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355c4d36-6aea-4564-9879-6e0c1071a818 service swift] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:14:58.131977 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355c4d36-6aea-4564-9879-6e0c1071a818 service swift] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:14:58.132424 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355c4d36-6aea-4564-9879-6e0c1071a818 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:14:58.142667 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355c4d36-6aea-4564-9879-6e0c1071a818 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:14:58.144998 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355c4d36-6aea-4564-9879-6e0c1071a818 service swift] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:14:58.157263 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1267/2525] 10.222.0.59 () {60 vars in 1384 bytes} [Tue Oct 11 12:14:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 11 12:15:05.253128 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8123faf-3a04-47a4-bd67-93311aed5003 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:15:05.253647 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8123faf-3a04-47a4-bd67-93311aed5003 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:15:05.253647 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8123faf-3a04-47a4-bd67-93311aed5003 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:15:05.275352 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-c8123faf-3a04-47a4-bd67-93311aed5003 None None] MFA Rules not processed for user `8a19eceb62ce4157a4c5fe1866a89df9`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:15:05.276590 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-c8123faf-3a04-47a4-bd67-93311aed5003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:15:05.285566 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1259/2526] 10.222.0.59 () {58 vars in 967 bytes} [Tue Oct 11 12:15:05 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:15:05.298779 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355c4d36-6aea-4564-9879-6e0c1071a818 service swift] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:15:05.309800 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355c4d36-6aea-4564-9879-6e0c1071a818 service swift] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:15:05.315926 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89fa7f1-880f-450f-8863-d328054c8e6b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76e7fee1993a40d5ba4295e7daeb3e29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:15:05.316540 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89fa7f1-880f-450f-8863-d328054c8e6b service swift] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:15:05.316821 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89fa7f1-880f-450f-8863-d328054c8e6b service swift] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:15:05.317100 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89fa7f1-880f-450f-8863-d328054c8e6b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:15:05.325742 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89fa7f1-880f-450f-8863-d328054c8e6b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:15:05.327456 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89fa7f1-880f-450f-8863-d328054c8e6b service swift] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:15:05.335802 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1268/2527] 10.222.0.59 () {60 vars in 1384 bytes} [Tue Oct 11 12:15:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 11 12:15:11.889617 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8123faf-3a04-47a4-bd67-93311aed5003 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:15:11.891676 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-c8123faf-3a04-47a4-bd67-93311aed5003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:15:11.969478 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8123faf-3a04-47a4-bd67-93311aed5003 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:15:11.974186 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c02593-5afb-4f5a-a134-7f07ee67b133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:15:11.974958 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c02593-5afb-4f5a-a134-7f07ee67b133 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:15:11.975300 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c02593-5afb-4f5a-a134-7f07ee67b133 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:15:11.975633 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c02593-5afb-4f5a-a134-7f07ee67b133 admin admin] PATH_INFO: `/v3/users/33ade5d60ccb44bab2e8ee2f0b21e456` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:15:11.976789 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c02593-5afb-4f5a-a134-7f07ee67b133 admin admin] RBAC: Authorizing `identity:delete_user(user_id=33ade5d60ccb44bab2e8ee2f0b21e456)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:15:11.978829 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c02593-5afb-4f5a-a134-7f07ee67b133 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:15:12.036294 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-b0c02593-5afb-4f5a-a134-7f07ee67b133 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33ade5d60ccb44bab2e8ee2f0b21e456'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:15:12.049472 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-b0c02593-5afb-4f5a-a134-7f07ee67b133 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33ade5d60ccb44bab2e8ee2f0b21e456'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:15:12.061152 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-b0c02593-5afb-4f5a-a134-7f07ee67b133 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33ade5d60ccb44bab2e8ee2f0b21e456'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:15:12.062093 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-b0c02593-5afb-4f5a-a134-7f07ee67b133 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33ade5d60ccb44bab2e8ee2f0b21e456'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:15:12.073811 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-b0c02593-5afb-4f5a-a134-7f07ee67b133 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33ade5d60ccb44bab2e8ee2f0b21e456'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:15:12.080897 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1260/2528] 10.222.0.59 () {58 vars in 1224 bytes} [Tue Oct 11 12:15:11 2022] DELETE /identity/v3/users/33ade5d60ccb44bab2e8ee2f0b21e456 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:15:12.275890 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89fa7f1-880f-450f-8863-d328054c8e6b service swift] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:15:12.292555 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89fa7f1-880f-450f-8863-d328054c8e6b service swift] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:15:12.352296 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b232a6f-cbe0-4b16-bbea-b798fefd0f37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:15:12.352931 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b232a6f-cbe0-4b16-bbea-b798fefd0f37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:15:12.353203 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b232a6f-cbe0-4b16-bbea-b798fefd0f37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:15:12.353472 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b232a6f-cbe0-4b16-bbea-b798fefd0f37 admin admin] PATH_INFO: `/v3/projects/f48db192fac04a06932590c23b751595` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:15:12.354626 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b232a6f-cbe0-4b16-bbea-b798fefd0f37 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f48db192fac04a06932590c23b751595)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:15:12.356576 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b232a6f-cbe0-4b16-bbea-b798fefd0f37 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:15:12.405180 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-7b232a6f-cbe0-4b16-bbea-b798fefd0f37 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f48db192fac04a06932590c23b751595', 'request_id': 'req-7b232a6f-cbe0-4b16-bbea-b798fefd0f37'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:15:12.410875 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-7b232a6f-cbe0-4b16-bbea-b798fefd0f37 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f48db192fac04a06932590c23b751595', 'request_id': 'req-7b232a6f-cbe0-4b16-bbea-b798fefd0f37'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:15:12.417061 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-7b232a6f-cbe0-4b16-bbea-b798fefd0f37 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f48db192fac04a06932590c23b751595', 'request_id': 'req-7b232a6f-cbe0-4b16-bbea-b798fefd0f37'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:15:12.432719 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-7b232a6f-cbe0-4b16-bbea-b798fefd0f37 admin admin] The token cache is being invalidate because project f48db192fac04a06932590c23b751595 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81432) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 12:15:12.433015 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-7b232a6f-cbe0-4b16-bbea-b798fefd0f37 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:15:12.433653 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-7b232a6f-cbe0-4b16-bbea-b798fefd0f37 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:15:12.435512 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1269/2529] 10.222.0.59 () {58 vars in 1233 bytes} [Tue Oct 11 12:15:12 2022] DELETE /identity/v3/projects/f48db192fac04a06932590c23b751595 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:15:12.439630 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c02593-5afb-4f5a-a134-7f07ee67b133 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:15:12.441215 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-b0c02593-5afb-4f5a-a134-7f07ee67b133 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:15:12.454472 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c02593-5afb-4f5a-a134-7f07ee67b133 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:15:12.503337 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:15:12.513633 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3c51d1-692f-4a28-ab2f-afcbc9fc6452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:15:12.514274 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3c51d1-692f-4a28-ab2f-afcbc9fc6452 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:15:12.514547 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3c51d1-692f-4a28-ab2f-afcbc9fc6452 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:15:12.514844 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3c51d1-692f-4a28-ab2f-afcbc9fc6452 admin admin] PATH_INFO: `/v3/users/cf8bd4dbc2f144aeaadeb8a819c5d2c4` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:15:12.515961 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3c51d1-692f-4a28-ab2f-afcbc9fc6452 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf8bd4dbc2f144aeaadeb8a819c5d2c4)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:15:12.517901 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3c51d1-692f-4a28-ab2f-afcbc9fc6452 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:15:12.571203 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-3d3c51d1-692f-4a28-ab2f-afcbc9fc6452 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf8bd4dbc2f144aeaadeb8a819c5d2c4'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:15:12.583640 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-3d3c51d1-692f-4a28-ab2f-afcbc9fc6452 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf8bd4dbc2f144aeaadeb8a819c5d2c4'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:15:12.592679 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-3d3c51d1-692f-4a28-ab2f-afcbc9fc6452 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf8bd4dbc2f144aeaadeb8a819c5d2c4'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:15:12.593790 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-3d3c51d1-692f-4a28-ab2f-afcbc9fc6452 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf8bd4dbc2f144aeaadeb8a819c5d2c4'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:15:12.602080 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-3d3c51d1-692f-4a28-ab2f-afcbc9fc6452 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf8bd4dbc2f144aeaadeb8a819c5d2c4'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:15:12.610427 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1261/2530] 10.222.0.59 () {58 vars in 1224 bytes} [Tue Oct 11 12:15:12 2022] DELETE /identity/v3/users/cf8bd4dbc2f144aeaadeb8a819c5d2c4 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:15:12.787624 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b232a6f-cbe0-4b16-bbea-b798fefd0f37 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:15:12.789079 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-7b232a6f-cbe0-4b16-bbea-b798fefd0f37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:15:12.802350 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b232a6f-cbe0-4b16-bbea-b798fefd0f37 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:15:12.855480 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:15:12.869232 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d0703a-52e2-4741-8591-cd5ad966b64c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:15:12.869899 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d0703a-52e2-4741-8591-cd5ad966b64c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:15:12.870164 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d0703a-52e2-4741-8591-cd5ad966b64c admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:15:12.870437 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d0703a-52e2-4741-8591-cd5ad966b64c admin admin] PATH_INFO: `/v3/projects/a6fbe903c2b545ca997d121e213f4611` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:15:12.871718 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d0703a-52e2-4741-8591-cd5ad966b64c admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6fbe903c2b545ca997d121e213f4611)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:15:12.873639 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d0703a-52e2-4741-8591-cd5ad966b64c admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:15:12.924229 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-37d0703a-52e2-4741-8591-cd5ad966b64c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6fbe903c2b545ca997d121e213f4611', 'request_id': 'req-37d0703a-52e2-4741-8591-cd5ad966b64c'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:15:12.930188 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-37d0703a-52e2-4741-8591-cd5ad966b64c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6fbe903c2b545ca997d121e213f4611', 'request_id': 'req-37d0703a-52e2-4741-8591-cd5ad966b64c'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:15:12.937382 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-37d0703a-52e2-4741-8591-cd5ad966b64c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6fbe903c2b545ca997d121e213f4611', 'request_id': 'req-37d0703a-52e2-4741-8591-cd5ad966b64c'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:15:12.952074 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-37d0703a-52e2-4741-8591-cd5ad966b64c admin admin] The token cache is being invalidate because project a6fbe903c2b545ca997d121e213f4611 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81432) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 12:15:12.952365 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-37d0703a-52e2-4741-8591-cd5ad966b64c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:15:12.952996 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-37d0703a-52e2-4741-8591-cd5ad966b64c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:15:12.954493 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1270/2531] 10.222.0.59 () {58 vars in 1233 bytes} [Tue Oct 11 12:15:12 2022] DELETE /identity/v3/projects/a6fbe903c2b545ca997d121e213f4611 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:15:22.605504 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=29,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:15:22.948801 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=28,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:15:54.747710 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3c51d1-692f-4a28-ab2f-afcbc9fc6452 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:15:54.749269 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-3d3c51d1-692f-4a28-ab2f-afcbc9fc6452 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:15:54.809918 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3c51d1-692f-4a28-ab2f-afcbc9fc6452 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:15:54.860259 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c39ba19-b811-4d34-93da-3df372e9ce4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e563fe174db4cc19ae6bf8162cc3d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:15:54.860804 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c39ba19-b811-4d34-93da-3df372e9ce4f service neutron] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:15:54.861110 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c39ba19-b811-4d34-93da-3df372e9ce4f service neutron] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:15:54.861388 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c39ba19-b811-4d34-93da-3df372e9ce4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:15:54.863369 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-7c39ba19-b811-4d34-93da-3df372e9ce4f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:15:54.927880 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c39ba19-b811-4d34-93da-3df372e9ce4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:15:54.929326 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c39ba19-b811-4d34-93da-3df372e9ce4f service neutron] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:15:54.939406 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1262/2532] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:15:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:15:57.786051 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d0703a-52e2-4741-8591-cd5ad966b64c admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:15:57.787649 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-37d0703a-52e2-4741-8591-cd5ad966b64c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:15:57.801606 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d0703a-52e2-4741-8591-cd5ad966b64c admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:15:57.806385 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cca1c3-528a-4f78-befa-b9604b9b450a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:15:57.807050 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cca1c3-528a-4f78-befa-b9604b9b450a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:15:57.807331 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cca1c3-528a-4f78-befa-b9604b9b450a admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:15:57.807603 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cca1c3-528a-4f78-befa-b9604b9b450a admin admin] PATH_INFO: `/v3/users/abfa19ab0a624de6ab082e8b1e57a220` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:15:57.808705 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cca1c3-528a-4f78-befa-b9604b9b450a admin admin] RBAC: Authorizing `identity:delete_user(user_id=abfa19ab0a624de6ab082e8b1e57a220)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:15:57.810738 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cca1c3-528a-4f78-befa-b9604b9b450a admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:15:57.858289 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-b8cca1c3-528a-4f78-befa-b9604b9b450a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'abfa19ab0a624de6ab082e8b1e57a220'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:15:57.870399 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-b8cca1c3-528a-4f78-befa-b9604b9b450a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'abfa19ab0a624de6ab082e8b1e57a220'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:15:57.871089 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-b8cca1c3-528a-4f78-befa-b9604b9b450a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'abfa19ab0a624de6ab082e8b1e57a220'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:15:57.879159 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-b8cca1c3-528a-4f78-befa-b9604b9b450a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'abfa19ab0a624de6ab082e8b1e57a220'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:15:57.879780 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-b8cca1c3-528a-4f78-befa-b9604b9b450a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'abfa19ab0a624de6ab082e8b1e57a220'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:15:57.891898 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1271/2533] 10.222.0.59 () {58 vars in 1224 bytes} [Tue Oct 11 12:15:57 2022] DELETE /identity/v3/users/abfa19ab0a624de6ab082e8b1e57a220 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:15:58.068477 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c39ba19-b811-4d34-93da-3df372e9ce4f service neutron] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:15:58.081441 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c39ba19-b811-4d34-93da-3df372e9ce4f service neutron] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:15:58.141212 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6339a867-0658-4de3-bb32-b21deda8e7fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:15:58.141963 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6339a867-0658-4de3-bb32-b21deda8e7fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:15:58.142376 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6339a867-0658-4de3-bb32-b21deda8e7fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:15:58.142634 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6339a867-0658-4de3-bb32-b21deda8e7fe admin admin] PATH_INFO: `/v3/projects/fee1ecae514e4c79a4f0fe601391971d` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:15:58.143947 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6339a867-0658-4de3-bb32-b21deda8e7fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=fee1ecae514e4c79a4f0fe601391971d)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:15:58.146330 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6339a867-0658-4de3-bb32-b21deda8e7fe admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:15:58.154819 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=100 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:15:58.194966 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-6339a867-0658-4de3-bb32-b21deda8e7fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fee1ecae514e4c79a4f0fe601391971d', 'request_id': 'req-6339a867-0658-4de3-bb32-b21deda8e7fe'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:15:58.200416 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-6339a867-0658-4de3-bb32-b21deda8e7fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fee1ecae514e4c79a4f0fe601391971d', 'request_id': 'req-6339a867-0658-4de3-bb32-b21deda8e7fe'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:15:58.208059 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-6339a867-0658-4de3-bb32-b21deda8e7fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fee1ecae514e4c79a4f0fe601391971d', 'request_id': 'req-6339a867-0658-4de3-bb32-b21deda8e7fe'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:15:58.219833 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-6339a867-0658-4de3-bb32-b21deda8e7fe admin admin] The token cache is being invalidate because project fee1ecae514e4c79a4f0fe601391971d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 12:15:58.220400 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-6339a867-0658-4de3-bb32-b21deda8e7fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:15:58.221115 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-6339a867-0658-4de3-bb32-b21deda8e7fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:15:58.222626 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1263/2534] 10.222.0.59 () {58 vars in 1233 bytes} [Tue Oct 11 12:15:58 2022] DELETE /identity/v3/projects/fee1ecae514e4c79a4f0fe601391971d => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 12:15:58.227631 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53061e87-715d-42b4-a448-36ee269d2ba1 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:15:58.227915 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53061e87-715d-42b4-a448-36ee269d2ba1 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:15:58.228187 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53061e87-715d-42b4-a448-36ee269d2ba1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:15:58.248745 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-53061e87-715d-42b4-a448-36ee269d2ba1 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:15:58.249871 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-53061e87-715d-42b4-a448-36ee269d2ba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:15:58.311073 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1272/2535] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:15:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:15:58.319400 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe35eca-56ed-4a00-87ae-934e3721f476 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:15:58.319687 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe35eca-56ed-4a00-87ae-934e3721f476 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:15:58.320011 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe35eca-56ed-4a00-87ae-934e3721f476 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:15:58.341284 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-4fe35eca-56ed-4a00-87ae-934e3721f476 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:15:58.342486 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-4fe35eca-56ed-4a00-87ae-934e3721f476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:15:58.351637 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1264/2536] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:15:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:15:58.356091 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53061e87-715d-42b4-a448-36ee269d2ba1 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:15:58.366161 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53061e87-715d-42b4-a448-36ee269d2ba1 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:15:58.372043 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463c9cdb-4354-4d90-a9cb-177cb76a383b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:15:58.372637 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463c9cdb-4354-4d90-a9cb-177cb76a383b admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:15:58.372911 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463c9cdb-4354-4d90-a9cb-177cb76a383b admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:15:58.373180 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463c9cdb-4354-4d90-a9cb-177cb76a383b admin admin] PATH_INFO: `/v3/domains` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:15:58.373819 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463c9cdb-4354-4d90-a9cb-177cb76a383b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 12:15:58.374090 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463c9cdb-4354-4d90-a9cb-177cb76a383b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:15:58.375718 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463c9cdb-4354-4d90-a9cb-177cb76a383b admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:15:58.387649 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1273/2537] 10.222.0.59 () {58 vars in 1153 bytes} [Tue Oct 11 12:15:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 12:15:58.392303 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe35eca-56ed-4a00-87ae-934e3721f476 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:15:58.402843 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe35eca-56ed-4a00-87ae-934e3721f476 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:15:58.408907 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc873a2d-f6bd-4b22-b755-4f9b74827dc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:15:58.409516 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc873a2d-f6bd-4b22-b755-4f9b74827dc1 admin admin] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:15:58.409798 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc873a2d-f6bd-4b22-b755-4f9b74827dc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:15:58.410067 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc873a2d-f6bd-4b22-b755-4f9b74827dc1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:15:58.410737 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc873a2d-f6bd-4b22-b755-4f9b74827dc1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:15:58.412672 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc873a2d-f6bd-4b22-b755-4f9b74827dc1 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:15:58.444738 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1265/2538] 10.222.0.59 () {60 vars in 1153 bytes} [Tue Oct 11 12:15:58 2022] POST /identity/v3/projects => generated 375 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:15:58.449171 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463c9cdb-4354-4d90-a9cb-177cb76a383b admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:15:58.459608 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463c9cdb-4354-4d90-a9cb-177cb76a383b admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:15:58.519588 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b18e349-5c15-4d41-a162-64fd9a5c6a0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:15:58.520474 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b18e349-5c15-4d41-a162-64fd9a5c6a0f admin admin] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:15:58.520745 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b18e349-5c15-4d41-a162-64fd9a5c6a0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:15:58.521006 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b18e349-5c15-4d41-a162-64fd9a5c6a0f admin admin] PATH_INFO: `/v3/users` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:15:58.521654 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b18e349-5c15-4d41-a162-64fd9a5c6a0f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:15:58.523652 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b18e349-5c15-4d41-a162-64fd9a5c6a0f admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:15:58.531750 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:15:58.548026 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1274/2539] 10.222.0.59 () {60 vars in 1144 bytes} [Tue Oct 11 12:15:58 2022] POST /identity/v3/users => generated 288 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:15:58.552235 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc873a2d-f6bd-4b22-b755-4f9b74827dc1 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:15:58.563189 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc873a2d-f6bd-4b22-b755-4f9b74827dc1 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:15:58.568324 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de229817-3551-4303-922c-c5a08c99b435 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:15:58.568918 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de229817-3551-4303-922c-c5a08c99b435 admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:15:58.569192 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de229817-3551-4303-922c-c5a08c99b435 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:15:58.569480 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de229817-3551-4303-922c-c5a08c99b435 admin admin] PATH_INFO: `/v3/roles` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:15:58.570091 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de229817-3551-4303-922c-c5a08c99b435 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:15:58.571701 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de229817-3551-4303-922c-c5a08c99b435 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:15:58.581614 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1266/2540] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:15:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:15:58.585667 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b18e349-5c15-4d41-a162-64fd9a5c6a0f admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:15:58.596237 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b18e349-5c15-4d41-a162-64fd9a5c6a0f admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:15:58.601965 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e853d423-a4d3-45d3-ab6a-4e1f3dbeadcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:15:58.602573 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e853d423-a4d3-45d3-ab6a-4e1f3dbeadcf admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:15:58.602870 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e853d423-a4d3-45d3-ab6a-4e1f3dbeadcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:15:58.603138 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e853d423-a4d3-45d3-ab6a-4e1f3dbeadcf admin admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:15:58.603770 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e853d423-a4d3-45d3-ab6a-4e1f3dbeadcf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:15:58.605379 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e853d423-a4d3-45d3-ab6a-4e1f3dbeadcf admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:15:58.616316 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1275/2541] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:15:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:15:58.620325 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de229817-3551-4303-922c-c5a08c99b435 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:15:58.631095 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de229817-3551-4303-922c-c5a08c99b435 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:15:58.636011 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579f2bbc-d75b-48be-ae22-f0491788a202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:15:58.636677 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579f2bbc-d75b-48be-ae22-f0491788a202 admin admin] REQUEST_METHOD: `PUT` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:15:58.637058 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579f2bbc-d75b-48be-ae22-f0491788a202 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:15:58.637367 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579f2bbc-d75b-48be-ae22-f0491788a202 admin admin] PATH_INFO: `/v3/projects/1b6e91ae5ba7427fb12d19ca65ce686e/users/ead246dee1ef40ca92ebaf20186c5b82/roles/de2d2bd26fff4748a79990f97a107164` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:15:58.652915 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579f2bbc-d75b-48be-ae22-f0491788a202 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b6e91ae5ba7427fb12d19ca65ce686e, user_id=ead246dee1ef40ca92ebaf20186c5b82, role_id=de2d2bd26fff4748a79990f97a107164)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:15:58.655659 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579f2bbc-d75b-48be-ae22-f0491788a202 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:15:58.669732 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1267/2542] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 12:15:58 2022] PUT /identity/v3/projects/1b6e91ae5ba7427fb12d19ca65ce686e/users/ead246dee1ef40ca92ebaf20186c5b82/roles/de2d2bd26fff4748a79990f97a107164 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:15:58.680319 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec9d07b-fd3d-4a50-8997-f37adc1e0777 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:15:58.680614 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec9d07b-fd3d-4a50-8997-f37adc1e0777 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:15:58.680891 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec9d07b-fd3d-4a50-8997-f37adc1e0777 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:15:58.711387 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-3ec9d07b-fd3d-4a50-8997-f37adc1e0777 None None] MFA Rules not processed for user `ead246dee1ef40ca92ebaf20186c5b82`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:15:58.712635 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-3ec9d07b-fd3d-4a50-8997-f37adc1e0777 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:15:58.777719 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1276/2543] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:15:58 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:15:58.789422 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579f2bbc-d75b-48be-ae22-f0491788a202 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:15:58.791288 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-579f2bbc-d75b-48be-ae22-f0491788a202 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:15:58.867717 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579f2bbc-d75b-48be-ae22-f0491788a202 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:15:58.873797 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11f5fab-e8fe-436a-916b-44304b12317e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bad5e1cb306b46818096940acc3810b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:15:58.874404 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11f5fab-e8fe-436a-916b-44304b12317e service nova] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:15:58.874721 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11f5fab-e8fe-436a-916b-44304b12317e service nova] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:15:58.874995 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11f5fab-e8fe-436a-916b-44304b12317e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:15:58.882879 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11f5fab-e8fe-436a-916b-44304b12317e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:15:58.884444 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11f5fab-e8fe-436a-916b-44304b12317e service nova] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:15:58.895757 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1268/2544] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:15:58 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:15:59.113272 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec9d07b-fd3d-4a50-8997-f37adc1e0777 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:15:59.184473 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec9d07b-fd3d-4a50-8997-f37adc1e0777 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:15:59.190339 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea710547-01b1-46cb-a244-37a89285179f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:15:59.191002 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea710547-01b1-46cb-a244-37a89285179f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:15:59.191275 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea710547-01b1-46cb-a244-37a89285179f admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:15:59.191541 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea710547-01b1-46cb-a244-37a89285179f admin admin] PATH_INFO: `/v3/users/ead246dee1ef40ca92ebaf20186c5b82` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:15:59.192799 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea710547-01b1-46cb-a244-37a89285179f admin admin] RBAC: Authorizing `identity:delete_user(user_id=ead246dee1ef40ca92ebaf20186c5b82)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:15:59.194782 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea710547-01b1-46cb-a244-37a89285179f admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:15:59.249562 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-ea710547-01b1-46cb-a244-37a89285179f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ead246dee1ef40ca92ebaf20186c5b82'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:15:59.261438 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-ea710547-01b1-46cb-a244-37a89285179f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ead246dee1ef40ca92ebaf20186c5b82'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:15:59.262115 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-ea710547-01b1-46cb-a244-37a89285179f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ead246dee1ef40ca92ebaf20186c5b82'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:15:59.265701 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:INSERT=4,keystone:SELECT=86,keystone:DELETE=10 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:15:59.270596 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-ea710547-01b1-46cb-a244-37a89285179f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ead246dee1ef40ca92ebaf20186c5b82'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:15:59.271289 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-ea710547-01b1-46cb-a244-37a89285179f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ead246dee1ef40ca92ebaf20186c5b82'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:15:59.284846 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1277/2545] 10.222.0.59 () {58 vars in 1224 bytes} [Tue Oct 11 12:15:59 2022] DELETE /identity/v3/users/ead246dee1ef40ca92ebaf20186c5b82 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:15:59.289351 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c137f2-3a77-49c9-b368-0d809df8437f None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:15:59.289858 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c137f2-3a77-49c9-b368-0d809df8437f None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:15:59.289960 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c137f2-3a77-49c9-b368-0d809df8437f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:15:59.309289 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-37c137f2-3a77-49c9-b368-0d809df8437f None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:15:59.310503 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-37c137f2-3a77-49c9-b368-0d809df8437f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:15:59.323355 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:15:59.373119 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1269/2546] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:15:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:15:59.382951 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea710547-01b1-46cb-a244-37a89285179f admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:15:59.384689 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-ea710547-01b1-46cb-a244-37a89285179f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:15:59.443129 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea710547-01b1-46cb-a244-37a89285179f admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:15:59.449629 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227a8d66-a2ca-4a25-a71a-1dabb231cce7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e563fe174db4cc19ae6bf8162cc3d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:15:59.450241 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227a8d66-a2ca-4a25-a71a-1dabb231cce7 service neutron] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:15:59.450516 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227a8d66-a2ca-4a25-a71a-1dabb231cce7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:15:59.450845 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227a8d66-a2ca-4a25-a71a-1dabb231cce7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:15:59.460929 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227a8d66-a2ca-4a25-a71a-1dabb231cce7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:15:59.462368 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227a8d66-a2ca-4a25-a71a-1dabb231cce7 service neutron] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:15:59.472199 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1278/2547] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:15:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:15:59.746783 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c137f2-3a77-49c9-b368-0d809df8437f None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:15:59.748253 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-37c137f2-3a77-49c9-b368-0d809df8437f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:15:59.760850 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c137f2-3a77-49c9-b368-0d809df8437f None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:15:59.767286 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57697f5a-9774-46c4-8c3e-94e757dcb51f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:15:59.767929 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57697f5a-9774-46c4-8c3e-94e757dcb51f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:15:59.768222 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57697f5a-9774-46c4-8c3e-94e757dcb51f admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:15:59.768570 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57697f5a-9774-46c4-8c3e-94e757dcb51f admin admin] PATH_INFO: `/v3/projects/1b6e91ae5ba7427fb12d19ca65ce686e` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:15:59.770525 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57697f5a-9774-46c4-8c3e-94e757dcb51f admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b6e91ae5ba7427fb12d19ca65ce686e)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:15:59.772676 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57697f5a-9774-46c4-8c3e-94e757dcb51f admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:15:59.822520 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-57697f5a-9774-46c4-8c3e-94e757dcb51f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b6e91ae5ba7427fb12d19ca65ce686e', 'request_id': 'req-57697f5a-9774-46c4-8c3e-94e757dcb51f'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:15:59.828228 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-57697f5a-9774-46c4-8c3e-94e757dcb51f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b6e91ae5ba7427fb12d19ca65ce686e', 'request_id': 'req-57697f5a-9774-46c4-8c3e-94e757dcb51f'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:15:59.836723 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-57697f5a-9774-46c4-8c3e-94e757dcb51f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b6e91ae5ba7427fb12d19ca65ce686e', 'request_id': 'req-57697f5a-9774-46c4-8c3e-94e757dcb51f'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:15:59.848992 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-57697f5a-9774-46c4-8c3e-94e757dcb51f admin admin] The token cache is being invalidate because project 1b6e91ae5ba7427fb12d19ca65ce686e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81433) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 12:15:59.849282 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-57697f5a-9774-46c4-8c3e-94e757dcb51f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:15:59.850020 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-57697f5a-9774-46c4-8c3e-94e757dcb51f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:15:59.851647 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1270/2548] 10.222.0.59 () {58 vars in 1233 bytes} [Tue Oct 11 12:15:59 2022] DELETE /identity/v3/projects/1b6e91ae5ba7427fb12d19ca65ce686e => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:15:59.856746 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb72b44-07e5-4581-9225-852fa315ab20 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:15:59.857053 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb72b44-07e5-4581-9225-852fa315ab20 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:15:59.857329 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb72b44-07e5-4581-9225-852fa315ab20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:15:59.877053 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-8fb72b44-07e5-4581-9225-852fa315ab20 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:15:59.878237 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-8fb72b44-07e5-4581-9225-852fa315ab20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:15:59.940124 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1279/2549] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:15:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:15:59.949614 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bddffd-aaa5-4bc8-a780-1e68a1bb0646 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:15:59.949886 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bddffd-aaa5-4bc8-a780-1e68a1bb0646 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:15:59.950162 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bddffd-aaa5-4bc8-a780-1e68a1bb0646 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:15:59.970806 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-16bddffd-aaa5-4bc8-a780-1e68a1bb0646 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:15:59.972016 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-16bddffd-aaa5-4bc8-a780-1e68a1bb0646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:15:59.980569 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1271/2550] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:15:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:15:59.984993 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb72b44-07e5-4581-9225-852fa315ab20 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:15:59.994857 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb72b44-07e5-4581-9225-852fa315ab20 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:16:00.000735 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb702e9-fe2d-47d4-9473-13d811bd3e33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:16:00.001363 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb702e9-fe2d-47d4-9473-13d811bd3e33 admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:16:00.001672 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb702e9-fe2d-47d4-9473-13d811bd3e33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:16:00.001981 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb702e9-fe2d-47d4-9473-13d811bd3e33 admin admin] PATH_INFO: `/v3/domains` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:16:00.002621 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb702e9-fe2d-47d4-9473-13d811bd3e33 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81432) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 12:16:00.002940 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb702e9-fe2d-47d4-9473-13d811bd3e33 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:16:00.004583 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb702e9-fe2d-47d4-9473-13d811bd3e33 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:16:00.019148 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1280/2551] 10.222.0.59 () {58 vars in 1153 bytes} [Tue Oct 11 12:15:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 12:16:00.023751 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bddffd-aaa5-4bc8-a780-1e68a1bb0646 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:16:00.034593 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bddffd-aaa5-4bc8-a780-1e68a1bb0646 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:16:00.041003 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d23cb6-cae2-4c4d-b68d-088b75a077fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:16:00.041745 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d23cb6-cae2-4c4d-b68d-088b75a077fb admin admin] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:16:00.042119 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d23cb6-cae2-4c4d-b68d-088b75a077fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:16:00.042464 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d23cb6-cae2-4c4d-b68d-088b75a077fb admin admin] PATH_INFO: `/v3/projects` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:16:00.043195 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d23cb6-cae2-4c4d-b68d-088b75a077fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:16:00.045351 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d23cb6-cae2-4c4d-b68d-088b75a077fb admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:16:00.074804 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1272/2552] 10.222.0.59 () {60 vars in 1153 bytes} [Tue Oct 11 12:16:00 2022] POST /identity/v3/projects => generated 357 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:16:00.078869 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb702e9-fe2d-47d4-9473-13d811bd3e33 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:16:00.088844 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb702e9-fe2d-47d4-9473-13d811bd3e33 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:16:00.154896 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b3da42-a7c3-4e06-a3ab-904a9163c2a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:16:00.155590 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b3da42-a7c3-4e06-a3ab-904a9163c2a4 admin admin] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:16:00.155772 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b3da42-a7c3-4e06-a3ab-904a9163c2a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:16:00.156042 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b3da42-a7c3-4e06-a3ab-904a9163c2a4 admin admin] PATH_INFO: `/v3/users` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:16:00.156668 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b3da42-a7c3-4e06-a3ab-904a9163c2a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:16:00.158678 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b3da42-a7c3-4e06-a3ab-904a9163c2a4 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:16:00.181965 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1281/2553] 10.222.0.59 () {60 vars in 1144 bytes} [Tue Oct 11 12:16:00 2022] POST /identity/v3/users => generated 279 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:16:00.186488 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d23cb6-cae2-4c4d-b68d-088b75a077fb admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:16:00.197026 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d23cb6-cae2-4c4d-b68d-088b75a077fb admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:16:00.201767 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b348dea-857f-4cf3-adac-6c5ef18e920c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:16:00.202356 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b348dea-857f-4cf3-adac-6c5ef18e920c admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:16:00.202692 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b348dea-857f-4cf3-adac-6c5ef18e920c admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:16:00.202960 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b348dea-857f-4cf3-adac-6c5ef18e920c admin admin] PATH_INFO: `/v3/roles` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:16:00.203625 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b348dea-857f-4cf3-adac-6c5ef18e920c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:16:00.205205 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b348dea-857f-4cf3-adac-6c5ef18e920c admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:16:00.215540 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1273/2554] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:16:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:16:00.219666 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b3da42-a7c3-4e06-a3ab-904a9163c2a4 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:16:00.230242 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b3da42-a7c3-4e06-a3ab-904a9163c2a4 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:16:00.236109 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a583dd3e-2ca1-4ba4-8d23-810f3de7637f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:16:00.236713 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a583dd3e-2ca1-4ba4-8d23-810f3de7637f admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:16:00.236995 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a583dd3e-2ca1-4ba4-8d23-810f3de7637f admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:16:00.237269 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a583dd3e-2ca1-4ba4-8d23-810f3de7637f admin admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:16:00.237920 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a583dd3e-2ca1-4ba4-8d23-810f3de7637f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:16:00.239576 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a583dd3e-2ca1-4ba4-8d23-810f3de7637f admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:16:00.240892 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:16:00.251336 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1282/2555] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:16:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 11 12:16:00.255491 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b348dea-857f-4cf3-adac-6c5ef18e920c admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:16:00.265840 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b348dea-857f-4cf3-adac-6c5ef18e920c admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:16:00.271254 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37497ef5-f2c3-4cbe-9ac9-848f0549834d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:16:00.271905 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37497ef5-f2c3-4cbe-9ac9-848f0549834d admin admin] REQUEST_METHOD: `PUT` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:16:00.272188 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37497ef5-f2c3-4cbe-9ac9-848f0549834d admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:16:00.272462 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37497ef5-f2c3-4cbe-9ac9-848f0549834d admin admin] PATH_INFO: `/v3/projects/f46d7a90ed454871a00319c869ad0902/users/76cfa80ab5c046e6acc0ff1f93d9df08/roles/de2d2bd26fff4748a79990f97a107164` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:16:00.287251 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37497ef5-f2c3-4cbe-9ac9-848f0549834d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f46d7a90ed454871a00319c869ad0902, user_id=76cfa80ab5c046e6acc0ff1f93d9df08, role_id=de2d2bd26fff4748a79990f97a107164)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:16:00.289937 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37497ef5-f2c3-4cbe-9ac9-848f0549834d admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:16:00.304641 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1274/2556] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 12:16:00 2022] PUT /identity/v3/projects/f46d7a90ed454871a00319c869ad0902/users/76cfa80ab5c046e6acc0ff1f93d9df08/roles/de2d2bd26fff4748a79990f97a107164 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:16:00.315751 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a2810a-7041-4b6a-9a99-dd34f732023a None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:16:00.316094 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a2810a-7041-4b6a-9a99-dd34f732023a None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:16:00.316470 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a2810a-7041-4b6a-9a99-dd34f732023a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:16:00.348601 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-a6a2810a-7041-4b6a-9a99-dd34f732023a None None] MFA Rules not processed for user `76cfa80ab5c046e6acc0ff1f93d9df08`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:16:00.349738 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-a6a2810a-7041-4b6a-9a99-dd34f732023a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:16:00.415954 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1283/2557] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:16:00 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:16:00.420789 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37497ef5-f2c3-4cbe-9ac9-848f0549834d admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:16:00.462328 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:16:00.484463 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37497ef5-f2c3-4cbe-9ac9-848f0549834d admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:16:00.489599 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34610469-8466-44f5-81ec-d56e34c34182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:16:00.490183 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34610469-8466-44f5-81ec-d56e34c34182 admin admin] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:16:00.490459 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34610469-8466-44f5-81ec-d56e34c34182 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:16:00.490774 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34610469-8466-44f5-81ec-d56e34c34182 admin admin] PATH_INFO: `/v3/projects` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:16:00.491397 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34610469-8466-44f5-81ec-d56e34c34182 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:16:00.493287 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34610469-8466-44f5-81ec-d56e34c34182 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:16:00.521674 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1275/2558] 10.222.0.59 () {60 vars in 1153 bytes} [Tue Oct 11 12:16:00 2022] POST /identity/v3/projects => generated 355 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:16:00.525856 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a2810a-7041-4b6a-9a99-dd34f732023a None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:16:00.536701 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a2810a-7041-4b6a-9a99-dd34f732023a None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:16:00.596076 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47597e56-f167-4a5c-bca6-7b310b533e8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:16:00.596689 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47597e56-f167-4a5c-bca6-7b310b533e8c admin admin] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:16:00.596977 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47597e56-f167-4a5c-bca6-7b310b533e8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:16:00.597255 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47597e56-f167-4a5c-bca6-7b310b533e8c admin admin] PATH_INFO: `/v3/users` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:16:00.597885 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47597e56-f167-4a5c-bca6-7b310b533e8c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:16:00.599963 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47597e56-f167-4a5c-bca6-7b310b533e8c admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:16:00.624047 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1284/2559] 10.222.0.59 () {60 vars in 1144 bytes} [Tue Oct 11 12:16:00 2022] POST /identity/v3/users => generated 284 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:16:00.628166 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34610469-8466-44f5-81ec-d56e34c34182 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:16:00.638822 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34610469-8466-44f5-81ec-d56e34c34182 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:16:00.644309 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a948ecab-efd7-416c-a3a8-58891b2eca5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:16:00.645254 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a948ecab-efd7-416c-a3a8-58891b2eca5a admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:16:00.645546 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a948ecab-efd7-416c-a3a8-58891b2eca5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:16:00.645806 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a948ecab-efd7-416c-a3a8-58891b2eca5a admin admin] PATH_INFO: `/v3/roles` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:16:00.646446 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a948ecab-efd7-416c-a3a8-58891b2eca5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:16:00.648101 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a948ecab-efd7-416c-a3a8-58891b2eca5a admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:16:00.658865 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1276/2560] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:16:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:16:00.662858 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47597e56-f167-4a5c-bca6-7b310b533e8c admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:16:00.672804 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47597e56-f167-4a5c-bca6-7b310b533e8c admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:16:00.677637 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e233a5ea-60f1-4b3b-a7b0-3a46b350c48c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:16:00.678278 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e233a5ea-60f1-4b3b-a7b0-3a46b350c48c admin admin] REQUEST_METHOD: `PUT` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:16:00.678580 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e233a5ea-60f1-4b3b-a7b0-3a46b350c48c admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:16:00.678859 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e233a5ea-60f1-4b3b-a7b0-3a46b350c48c admin admin] PATH_INFO: `/v3/projects/a73266920d7f4454815dcf43650d7672/users/7af7501c0b014846b624ee4460fa440c/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:16:00.693887 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e233a5ea-60f1-4b3b-a7b0-3a46b350c48c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a73266920d7f4454815dcf43650d7672, user_id=7af7501c0b014846b624ee4460fa440c, role_id=5cf1f4bd2fef46ab9c6eff56d52f9f69)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:16:00.696570 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e233a5ea-60f1-4b3b-a7b0-3a46b350c48c admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:16:00.712080 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1285/2561] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 12:16:00 2022] PUT /identity/v3/projects/a73266920d7f4454815dcf43650d7672/users/7af7501c0b014846b624ee4460fa440c/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:16:00.716178 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a948ecab-efd7-416c-a3a8-58891b2eca5a admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:16:00.726769 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a948ecab-efd7-416c-a3a8-58891b2eca5a admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:16:00.786965 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74eeafe7-e037-4422-af47-7ae18263652b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:16:00.787560 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74eeafe7-e037-4422-af47-7ae18263652b admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:16:00.787823 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74eeafe7-e037-4422-af47-7ae18263652b admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:16:00.788109 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74eeafe7-e037-4422-af47-7ae18263652b admin admin] PATH_INFO: `/v3/roles` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:16:00.788781 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74eeafe7-e037-4422-af47-7ae18263652b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:16:00.790368 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74eeafe7-e037-4422-af47-7ae18263652b admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:16:00.804650 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1277/2562] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:16:00 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 11 12:16:00.809133 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e233a5ea-60f1-4b3b-a7b0-3a46b350c48c admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:16:00.819750 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e233a5ea-60f1-4b3b-a7b0-3a46b350c48c admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:16:00.824780 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d115f73-03f0-478b-b4fc-ee721fba698b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:16:00.825419 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d115f73-03f0-478b-b4fc-ee721fba698b admin admin] REQUEST_METHOD: `PUT` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:16:00.825700 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d115f73-03f0-478b-b4fc-ee721fba698b admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:16:00.825972 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d115f73-03f0-478b-b4fc-ee721fba698b admin admin] PATH_INFO: `/v3/projects/a73266920d7f4454815dcf43650d7672/users/7af7501c0b014846b624ee4460fa440c/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:16:00.827933 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d115f73-03f0-478b-b4fc-ee721fba698b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a73266920d7f4454815dcf43650d7672, user_id=7af7501c0b014846b624ee4460fa440c, role_id=5cf1f4bd2fef46ab9c6eff56d52f9f69)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:16:00.830594 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d115f73-03f0-478b-b4fc-ee721fba698b admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:16:00.840932 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1286/2563] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 12:16:00 2022] PUT /identity/v3/projects/a73266920d7f4454815dcf43650d7672/users/7af7501c0b014846b624ee4460fa440c/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:16:00.851483 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477caa50-abad-4d70-9ebc-813d69545457 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:16:00.851686 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477caa50-abad-4d70-9ebc-813d69545457 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:16:00.852037 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477caa50-abad-4d70-9ebc-813d69545457 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:16:00.883228 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-477caa50-abad-4d70-9ebc-813d69545457 None None] MFA Rules not processed for user `7af7501c0b014846b624ee4460fa440c`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:16:00.884607 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-477caa50-abad-4d70-9ebc-813d69545457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:16:00.954689 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1278/2564] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:16:00 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:16:00.966310 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d115f73-03f0-478b-b4fc-ee721fba698b admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:16:00.968130 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-9d115f73-03f0-478b-b4fc-ee721fba698b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:16:01.010463 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:16:01.026273 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d115f73-03f0-478b-b4fc-ee721fba698b admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:16:01.030266 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b16b47-53bd-4329-82fd-c17054b9f99f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b1b651c73664d08accc6edf2f79bace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:16:01.030898 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b16b47-53bd-4329-82fd-c17054b9f99f service cinder] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:16:01.031181 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b16b47-53bd-4329-82fd-c17054b9f99f service cinder] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:16:01.031449 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b16b47-53bd-4329-82fd-c17054b9f99f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:16:01.040585 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b16b47-53bd-4329-82fd-c17054b9f99f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:16:01.042016 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b16b47-53bd-4329-82fd-c17054b9f99f service cinder] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:16:01.051857 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1287/2565] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:16:00 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:16:01.146654 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477caa50-abad-4d70-9ebc-813d69545457 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:16:01.157334 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477caa50-abad-4d70-9ebc-813d69545457 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:16:01.161509 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4ab14c-198e-4289-8bec-dbe82254115d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b1b651c73664d08accc6edf2f79bace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:16:01.162110 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4ab14c-198e-4289-8bec-dbe82254115d service cinder] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:16:01.162424 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4ab14c-198e-4289-8bec-dbe82254115d service cinder] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:16:01.162727 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4ab14c-198e-4289-8bec-dbe82254115d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:16:01.193426 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:16:01.221056 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4ab14c-198e-4289-8bec-dbe82254115d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:16:01.222495 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4ab14c-198e-4289-8bec-dbe82254115d service cinder] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:16:01.231812 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1279/2566] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:16:01 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:16:04.512091 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b16b47-53bd-4329-82fd-c17054b9f99f service cinder] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:16:04.513598 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-00b16b47-53bd-4329-82fd-c17054b9f99f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:16:04.592629 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b16b47-53bd-4329-82fd-c17054b9f99f service cinder] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:16:04.598365 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f341c800-412e-45f3-9b4b-4d6d21a80c8b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bad5e1cb306b46818096940acc3810b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:16:04.599080 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f341c800-412e-45f3-9b4b-4d6d21a80c8b service nova] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:16:04.599359 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f341c800-412e-45f3-9b4b-4d6d21a80c8b service nova] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:16:04.599864 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f341c800-412e-45f3-9b4b-4d6d21a80c8b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:16:04.601749 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-f341c800-412e-45f3-9b4b-4d6d21a80c8b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:16:04.615361 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f341c800-412e-45f3-9b4b-4d6d21a80c8b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:16:04.616989 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f341c800-412e-45f3-9b4b-4d6d21a80c8b service nova] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:16:04.628335 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1288/2567] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:16:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:16:06.300458 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eae1288-4f93-41b8-a9df-b399c4a00ff1 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:16:06.300962 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eae1288-4f93-41b8-a9df-b399c4a00ff1 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:16:06.300962 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eae1288-4f93-41b8-a9df-b399c4a00ff1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:16:06.320123 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-5eae1288-4f93-41b8-a9df-b399c4a00ff1 None None] MFA Rules not processed for user `82c54e3aa3aa46a28373acb91f32cc9a`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:16:06.321314 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-5eae1288-4f93-41b8-a9df-b399c4a00ff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:16:06.378945 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1280/2568] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:16:06 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:16:08.933032 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f341c800-412e-45f3-9b4b-4d6d21a80c8b service nova] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:16:08.934657 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-f341c800-412e-45f3-9b4b-4d6d21a80c8b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:16:09.007328 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f341c800-412e-45f3-9b4b-4d6d21a80c8b service nova] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:16:09.010915 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c903599f-9283-4f51-b2ce-fa5e8a48e28a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e563fe174db4cc19ae6bf8162cc3d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:16:09.011528 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c903599f-9283-4f51-b2ce-fa5e8a48e28a service neutron] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:16:09.011836 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c903599f-9283-4f51-b2ce-fa5e8a48e28a service neutron] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:16:09.012108 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c903599f-9283-4f51-b2ce-fa5e8a48e28a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:16:09.013797 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-c903599f-9283-4f51-b2ce-fa5e8a48e28a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:16:09.023554 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c903599f-9283-4f51-b2ce-fa5e8a48e28a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:16:09.024976 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c903599f-9283-4f51-b2ce-fa5e8a48e28a service neutron] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:16:09.033055 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1289/2569] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:16:08 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:16:16.350111 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eae1288-4f93-41b8-a9df-b399c4a00ff1 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:16:16.413217 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eae1288-4f93-41b8-a9df-b399c4a00ff1 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:16:16.417879 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9292cc8-c564-4f7a-b5b8-6fa74e3c6c82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:16:16.418538 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9292cc8-c564-4f7a-b5b8-6fa74e3c6c82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:16:16.418854 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9292cc8-c564-4f7a-b5b8-6fa74e3c6c82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:16:16.419128 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9292cc8-c564-4f7a-b5b8-6fa74e3c6c82 admin admin] PATH_INFO: `/v3/users/76cfa80ab5c046e6acc0ff1f93d9df08` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:16:16.420396 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9292cc8-c564-4f7a-b5b8-6fa74e3c6c82 admin admin] RBAC: Authorizing `identity:delete_user(user_id=76cfa80ab5c046e6acc0ff1f93d9df08)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:16:16.422336 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9292cc8-c564-4f7a-b5b8-6fa74e3c6c82 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:16:16.470423 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-d9292cc8-c564-4f7a-b5b8-6fa74e3c6c82 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76cfa80ab5c046e6acc0ff1f93d9df08'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:16:16.482581 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-d9292cc8-c564-4f7a-b5b8-6fa74e3c6c82 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76cfa80ab5c046e6acc0ff1f93d9df08'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:16:16.491704 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-d9292cc8-c564-4f7a-b5b8-6fa74e3c6c82 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76cfa80ab5c046e6acc0ff1f93d9df08'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:16:16.492345 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-d9292cc8-c564-4f7a-b5b8-6fa74e3c6c82 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76cfa80ab5c046e6acc0ff1f93d9df08'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:16:16.497684 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:16:16.499679 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-d9292cc8-c564-4f7a-b5b8-6fa74e3c6c82 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76cfa80ab5c046e6acc0ff1f93d9df08'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:16:16.507229 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1281/2570] 10.222.0.59 () {58 vars in 1224 bytes} [Tue Oct 11 12:16:16 2022] DELETE /identity/v3/users/76cfa80ab5c046e6acc0ff1f93d9df08 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:16:16.511943 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75a6447-4a2d-4862-8454-367bf5522da0 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:16:16.512526 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75a6447-4a2d-4862-8454-367bf5522da0 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:16:16.512526 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75a6447-4a2d-4862-8454-367bf5522da0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:16:16.532288 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-c75a6447-4a2d-4862-8454-367bf5522da0 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:16:16.533484 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-c75a6447-4a2d-4862-8454-367bf5522da0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:16:16.599478 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1290/2571] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:16:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:16:16.609595 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9292cc8-c564-4f7a-b5b8-6fa74e3c6c82 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:16:16.611075 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-d9292cc8-c564-4f7a-b5b8-6fa74e3c6c82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:16:16.671167 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9292cc8-c564-4f7a-b5b8-6fa74e3c6c82 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:16:16.675157 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba75e68c-b38a-49ff-bdbc-eece7ffffc4a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e563fe174db4cc19ae6bf8162cc3d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:16:16.675769 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba75e68c-b38a-49ff-bdbc-eece7ffffc4a service neutron] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:16:16.676044 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba75e68c-b38a-49ff-bdbc-eece7ffffc4a service neutron] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:16:16.676312 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba75e68c-b38a-49ff-bdbc-eece7ffffc4a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:16:16.685993 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba75e68c-b38a-49ff-bdbc-eece7ffffc4a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:16:16.687398 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba75e68c-b38a-49ff-bdbc-eece7ffffc4a service neutron] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:16:16.696906 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1282/2572] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:16:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:16:16.972784 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75a6447-4a2d-4862-8454-367bf5522da0 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:16:16.974345 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-c75a6447-4a2d-4862-8454-367bf5522da0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:16:16.986756 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75a6447-4a2d-4862-8454-367bf5522da0 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:16:16.992500 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efadff5-28ca-4beb-b830-b9ab53ea5d01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:16:16.993151 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efadff5-28ca-4beb-b830-b9ab53ea5d01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:16:16.993426 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efadff5-28ca-4beb-b830-b9ab53ea5d01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:16:16.993693 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efadff5-28ca-4beb-b830-b9ab53ea5d01 admin admin] PATH_INFO: `/v3/projects/f46d7a90ed454871a00319c869ad0902` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:16:16.994984 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efadff5-28ca-4beb-b830-b9ab53ea5d01 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f46d7a90ed454871a00319c869ad0902)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:16:16.997055 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efadff5-28ca-4beb-b830-b9ab53ea5d01 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:16:16.999602 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=100 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:16:17.044830 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-4efadff5-28ca-4beb-b830-b9ab53ea5d01 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f46d7a90ed454871a00319c869ad0902', 'request_id': 'req-4efadff5-28ca-4beb-b830-b9ab53ea5d01'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:16:17.050243 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-4efadff5-28ca-4beb-b830-b9ab53ea5d01 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f46d7a90ed454871a00319c869ad0902', 'request_id': 'req-4efadff5-28ca-4beb-b830-b9ab53ea5d01'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:16:17.056497 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-4efadff5-28ca-4beb-b830-b9ab53ea5d01 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f46d7a90ed454871a00319c869ad0902', 'request_id': 'req-4efadff5-28ca-4beb-b830-b9ab53ea5d01'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:16:17.070890 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-4efadff5-28ca-4beb-b830-b9ab53ea5d01 admin admin] The token cache is being invalidate because project f46d7a90ed454871a00319c869ad0902 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81432) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 12:16:17.071170 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-4efadff5-28ca-4beb-b830-b9ab53ea5d01 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:16:17.072044 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-4efadff5-28ca-4beb-b830-b9ab53ea5d01 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:16:17.073593 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1291/2573] 10.222.0.59 () {58 vars in 1233 bytes} [Tue Oct 11 12:16:16 2022] DELETE /identity/v3/projects/f46d7a90ed454871a00319c869ad0902 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:16:17.077715 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba75e68c-b38a-49ff-bdbc-eece7ffffc4a service neutron] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:16:17.087609 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba75e68c-b38a-49ff-bdbc-eece7ffffc4a service neutron] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:16:17.145831 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb7a3bd-e0ed-4991-91f3-acc2e7978473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:16:17.146497 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb7a3bd-e0ed-4991-91f3-acc2e7978473 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:16:17.146799 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb7a3bd-e0ed-4991-91f3-acc2e7978473 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:16:17.147069 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb7a3bd-e0ed-4991-91f3-acc2e7978473 admin admin] PATH_INFO: `/v3/users/7af7501c0b014846b624ee4460fa440c` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:16:17.148282 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb7a3bd-e0ed-4991-91f3-acc2e7978473 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7af7501c0b014846b624ee4460fa440c)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:16:17.150308 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb7a3bd-e0ed-4991-91f3-acc2e7978473 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:16:17.203479 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-8bb7a3bd-e0ed-4991-91f3-acc2e7978473 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7af7501c0b014846b624ee4460fa440c'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:16:17.215904 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-8bb7a3bd-e0ed-4991-91f3-acc2e7978473 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7af7501c0b014846b624ee4460fa440c'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:16:17.225564 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-8bb7a3bd-e0ed-4991-91f3-acc2e7978473 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7af7501c0b014846b624ee4460fa440c'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:16:17.226205 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-8bb7a3bd-e0ed-4991-91f3-acc2e7978473 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7af7501c0b014846b624ee4460fa440c'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:16:17.234350 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-8bb7a3bd-e0ed-4991-91f3-acc2e7978473 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7af7501c0b014846b624ee4460fa440c'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:16:17.240442 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1283/2574] 10.222.0.59 () {58 vars in 1224 bytes} [Tue Oct 11 12:16:17 2022] DELETE /identity/v3/users/7af7501c0b014846b624ee4460fa440c => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:16:17.510082 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efadff5-28ca-4beb-b830-b9ab53ea5d01 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:16:17.511705 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-4efadff5-28ca-4beb-b830-b9ab53ea5d01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:16:17.527810 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efadff5-28ca-4beb-b830-b9ab53ea5d01 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:16:17.588923 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e904e60f-30a4-40f7-81e3-2b0b687936d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:16:17.589512 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e904e60f-30a4-40f7-81e3-2b0b687936d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:16:17.589792 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e904e60f-30a4-40f7-81e3-2b0b687936d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:16:17.590058 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e904e60f-30a4-40f7-81e3-2b0b687936d5 admin admin] PATH_INFO: `/v3/projects/a73266920d7f4454815dcf43650d7672` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:16:17.591178 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e904e60f-30a4-40f7-81e3-2b0b687936d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a73266920d7f4454815dcf43650d7672)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:16:17.593091 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e904e60f-30a4-40f7-81e3-2b0b687936d5 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:16:17.640960 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-e904e60f-30a4-40f7-81e3-2b0b687936d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a73266920d7f4454815dcf43650d7672', 'request_id': 'req-e904e60f-30a4-40f7-81e3-2b0b687936d5'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:16:17.647541 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-e904e60f-30a4-40f7-81e3-2b0b687936d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a73266920d7f4454815dcf43650d7672', 'request_id': 'req-e904e60f-30a4-40f7-81e3-2b0b687936d5'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:16:17.654022 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-e904e60f-30a4-40f7-81e3-2b0b687936d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a73266920d7f4454815dcf43650d7672', 'request_id': 'req-e904e60f-30a4-40f7-81e3-2b0b687936d5'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:16:17.668426 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-e904e60f-30a4-40f7-81e3-2b0b687936d5 admin admin] The token cache is being invalidate because project a73266920d7f4454815dcf43650d7672 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81432) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 12:16:17.668774 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-e904e60f-30a4-40f7-81e3-2b0b687936d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:16:17.669473 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-e904e60f-30a4-40f7-81e3-2b0b687936d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:16:17.671092 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1292/2575] 10.222.0.59 () {58 vars in 1233 bytes} [Tue Oct 11 12:16:17 2022] DELETE /identity/v3/projects/a73266920d7f4454815dcf43650d7672 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:16:17.676095 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4173fa9-06dd-4b4b-bdb0-3680b5199004 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:16:17.676599 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4173fa9-06dd-4b4b-bdb0-3680b5199004 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:16:17.676599 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4173fa9-06dd-4b4b-bdb0-3680b5199004 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:16:17.697129 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-a4173fa9-06dd-4b4b-bdb0-3680b5199004 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:16:17.698315 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-a4173fa9-06dd-4b4b-bdb0-3680b5199004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:16:17.711457 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:16:17.763877 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1284/2576] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:16:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 12:16:17.771800 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7d3587-e084-410a-ba66-b5396899964e None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:16:17.772190 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7d3587-e084-410a-ba66-b5396899964e None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:16:17.772467 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7d3587-e084-410a-ba66-b5396899964e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:16:17.792705 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-cf7d3587-e084-410a-ba66-b5396899964e None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:16:17.793933 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-cf7d3587-e084-410a-ba66-b5396899964e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:16:17.802084 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1293/2577] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:16:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:16:17.806332 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4173fa9-06dd-4b4b-bdb0-3680b5199004 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:16:17.816449 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4173fa9-06dd-4b4b-bdb0-3680b5199004 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:16:17.821099 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415f7ca5-e139-4dea-ac66-62e8a62a8137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:16:17.821700 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415f7ca5-e139-4dea-ac66-62e8a62a8137 admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:16:17.821975 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415f7ca5-e139-4dea-ac66-62e8a62a8137 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:16:17.822357 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415f7ca5-e139-4dea-ac66-62e8a62a8137 admin admin] PATH_INFO: `/v3/domains` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:16:17.823039 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415f7ca5-e139-4dea-ac66-62e8a62a8137 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 12:16:17.823342 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415f7ca5-e139-4dea-ac66-62e8a62a8137 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:16:17.825099 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415f7ca5-e139-4dea-ac66-62e8a62a8137 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:16:17.838593 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1285/2578] 10.222.0.59 () {58 vars in 1153 bytes} [Tue Oct 11 12:16:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 12:16:17.841540 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7d3587-e084-410a-ba66-b5396899964e None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:16:17.850831 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7d3587-e084-410a-ba66-b5396899964e None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:16:17.855602 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10c5399-a2c0-4cd6-bf45-e49012de867b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:16:17.856206 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10c5399-a2c0-4cd6-bf45-e49012de867b admin admin] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:16:17.856511 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10c5399-a2c0-4cd6-bf45-e49012de867b admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:16:17.856781 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10c5399-a2c0-4cd6-bf45-e49012de867b admin admin] PATH_INFO: `/v3/projects` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:16:17.857435 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10c5399-a2c0-4cd6-bf45-e49012de867b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:16:17.859381 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10c5399-a2c0-4cd6-bf45-e49012de867b admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:16:17.875440 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:16:17.897034 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1294/2579] 10.222.0.59 () {60 vars in 1153 bytes} [Tue Oct 11 12:16:17 2022] POST /identity/v3/projects => generated 365 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:16:17.901040 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415f7ca5-e139-4dea-ac66-62e8a62a8137 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:16:17.911056 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415f7ca5-e139-4dea-ac66-62e8a62a8137 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:16:17.972278 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8f405d-d0f2-4e1b-bbd3-5f920da8a7f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:16:17.972897 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8f405d-d0f2-4e1b-bbd3-5f920da8a7f5 admin admin] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:16:17.973172 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8f405d-d0f2-4e1b-bbd3-5f920da8a7f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:16:17.973455 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8f405d-d0f2-4e1b-bbd3-5f920da8a7f5 admin admin] PATH_INFO: `/v3/users` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:16:17.974106 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8f405d-d0f2-4e1b-bbd3-5f920da8a7f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:16:17.976051 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8f405d-d0f2-4e1b-bbd3-5f920da8a7f5 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:16:17.999697 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1286/2580] 10.222.0.59 () {60 vars in 1144 bytes} [Tue Oct 11 12:16:17 2022] POST /identity/v3/users => generated 283 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:16:18.004092 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10c5399-a2c0-4cd6-bf45-e49012de867b admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:16:18.014291 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10c5399-a2c0-4cd6-bf45-e49012de867b admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:16:18.019079 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90536a4d-48fb-40b6-9219-d47ff526672a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:16:18.019667 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90536a4d-48fb-40b6-9219-d47ff526672a admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:16:18.020349 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90536a4d-48fb-40b6-9219-d47ff526672a admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:16:18.020620 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90536a4d-48fb-40b6-9219-d47ff526672a admin admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:16:18.021257 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90536a4d-48fb-40b6-9219-d47ff526672a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:16:18.022883 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90536a4d-48fb-40b6-9219-d47ff526672a admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:16:18.033030 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1295/2581] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:16:18 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:16:18.036785 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8f405d-d0f2-4e1b-bbd3-5f920da8a7f5 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:16:18.047412 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8f405d-d0f2-4e1b-bbd3-5f920da8a7f5 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:16:18.053058 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdbaaed-7e23-4855-b14a-81aa566f7b93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:16:18.053648 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdbaaed-7e23-4855-b14a-81aa566f7b93 admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:16:18.053921 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdbaaed-7e23-4855-b14a-81aa566f7b93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:16:18.054189 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdbaaed-7e23-4855-b14a-81aa566f7b93 admin admin] PATH_INFO: `/v3/roles` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:16:18.054862 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdbaaed-7e23-4855-b14a-81aa566f7b93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:16:18.056469 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdbaaed-7e23-4855-b14a-81aa566f7b93 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:16:18.067475 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1287/2582] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:16:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 11 12:16:18.071872 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90536a4d-48fb-40b6-9219-d47ff526672a admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:16:18.082240 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90536a4d-48fb-40b6-9219-d47ff526672a admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:16:18.087507 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a05cd2-aff2-4e0e-ac4c-73286b00fb5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:16:18.088164 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a05cd2-aff2-4e0e-ac4c-73286b00fb5b admin admin] REQUEST_METHOD: `PUT` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:16:18.088423 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a05cd2-aff2-4e0e-ac4c-73286b00fb5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:16:18.088696 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a05cd2-aff2-4e0e-ac4c-73286b00fb5b admin admin] PATH_INFO: `/v3/projects/8dd4f509f7bc46c6afd05ec88982d75c/users/1314daa45047484bb4015043e71efa55/roles/de2d2bd26fff4748a79990f97a107164` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:16:18.103569 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a05cd2-aff2-4e0e-ac4c-73286b00fb5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=8dd4f509f7bc46c6afd05ec88982d75c, user_id=1314daa45047484bb4015043e71efa55, role_id=de2d2bd26fff4748a79990f97a107164)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:16:18.106222 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a05cd2-aff2-4e0e-ac4c-73286b00fb5b admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:16:18.121045 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1296/2583] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 12:16:18 2022] PUT /identity/v3/projects/8dd4f509f7bc46c6afd05ec88982d75c/users/1314daa45047484bb4015043e71efa55/roles/de2d2bd26fff4748a79990f97a107164 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:16:18.130880 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce127047-1ffb-4b1f-bde7-ccd7f1b74047 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:16:18.131166 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce127047-1ffb-4b1f-bde7-ccd7f1b74047 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:16:18.131434 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce127047-1ffb-4b1f-bde7-ccd7f1b74047 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:16:18.159541 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-ce127047-1ffb-4b1f-bde7-ccd7f1b74047 None None] MFA Rules not processed for user `1314daa45047484bb4015043e71efa55`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:16:18.160783 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-ce127047-1ffb-4b1f-bde7-ccd7f1b74047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:16:18.228651 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1288/2584] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:16:18 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:16:18.233252 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a05cd2-aff2-4e0e-ac4c-73286b00fb5b admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:16:18.299414 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a05cd2-aff2-4e0e-ac4c-73286b00fb5b admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:16:18.305008 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5496c9b-c893-42d0-a62d-7da47e097112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:16:18.305621 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5496c9b-c893-42d0-a62d-7da47e097112 admin admin] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:16:18.305897 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5496c9b-c893-42d0-a62d-7da47e097112 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:16:18.306167 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5496c9b-c893-42d0-a62d-7da47e097112 admin admin] PATH_INFO: `/v3/projects` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:16:18.306888 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5496c9b-c893-42d0-a62d-7da47e097112 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:16:18.308798 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5496c9b-c893-42d0-a62d-7da47e097112 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:16:18.341346 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1297/2585] 10.222.0.59 () {60 vars in 1153 bytes} [Tue Oct 11 12:16:18 2022] POST /identity/v3/projects => generated 365 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:16:18.345461 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce127047-1ffb-4b1f-bde7-ccd7f1b74047 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:16:18.357298 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce127047-1ffb-4b1f-bde7-ccd7f1b74047 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:16:18.367237 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:16:18.420875 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0053302b-9abf-4eb9-8e07-8181f343fe55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:16:18.421487 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0053302b-9abf-4eb9-8e07-8181f343fe55 admin admin] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:16:18.421750 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0053302b-9abf-4eb9-8e07-8181f343fe55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:16:18.422016 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0053302b-9abf-4eb9-8e07-8181f343fe55 admin admin] PATH_INFO: `/v3/users` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:16:18.422693 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0053302b-9abf-4eb9-8e07-8181f343fe55 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:16:18.424594 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0053302b-9abf-4eb9-8e07-8181f343fe55 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:16:18.449438 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1289/2586] 10.222.0.59 () {60 vars in 1144 bytes} [Tue Oct 11 12:16:18 2022] POST /identity/v3/users => generated 289 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:16:18.453545 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5496c9b-c893-42d0-a62d-7da47e097112 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:16:18.464518 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5496c9b-c893-42d0-a62d-7da47e097112 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:16:18.470211 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219032de-4b03-4855-b907-47d362c6eaee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:16:18.470860 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219032de-4b03-4855-b907-47d362c6eaee admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:16:18.471136 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219032de-4b03-4855-b907-47d362c6eaee admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:16:18.471408 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219032de-4b03-4855-b907-47d362c6eaee admin admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:16:18.472051 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219032de-4b03-4855-b907-47d362c6eaee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:16:18.473714 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219032de-4b03-4855-b907-47d362c6eaee admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:16:18.484698 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1298/2587] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:16:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:16:18.488888 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0053302b-9abf-4eb9-8e07-8181f343fe55 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:16:18.499681 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0053302b-9abf-4eb9-8e07-8181f343fe55 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:16:18.505701 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b929127e-d37a-43b9-946c-2c669afef6dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:16:18.506362 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b929127e-d37a-43b9-946c-2c669afef6dc admin admin] REQUEST_METHOD: `PUT` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:16:18.506690 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b929127e-d37a-43b9-946c-2c669afef6dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:16:18.506967 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b929127e-d37a-43b9-946c-2c669afef6dc admin admin] PATH_INFO: `/v3/projects/325b8a46dc2a4b63b7d2c8c082abfe56/users/6e03db0c522046bba6b5a7df84f4036b/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:16:18.523325 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b929127e-d37a-43b9-946c-2c669afef6dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=325b8a46dc2a4b63b7d2c8c082abfe56, user_id=6e03db0c522046bba6b5a7df84f4036b, role_id=5cf1f4bd2fef46ab9c6eff56d52f9f69)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:16:18.526031 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b929127e-d37a-43b9-946c-2c669afef6dc admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:16:18.541070 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1290/2588] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 12:16:18 2022] PUT /identity/v3/projects/325b8a46dc2a4b63b7d2c8c082abfe56/users/6e03db0c522046bba6b5a7df84f4036b/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 11 12:16:18.545161 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219032de-4b03-4855-b907-47d362c6eaee admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:16:18.555738 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219032de-4b03-4855-b907-47d362c6eaee admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:16:18.611224 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39fefb6-9aef-4d52-b768-c15b7e92490e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:16:18.611754 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39fefb6-9aef-4d52-b768-c15b7e92490e admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:16:18.612034 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39fefb6-9aef-4d52-b768-c15b7e92490e admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:16:18.612304 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39fefb6-9aef-4d52-b768-c15b7e92490e admin admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:16:18.612947 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39fefb6-9aef-4d52-b768-c15b7e92490e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:16:18.614550 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39fefb6-9aef-4d52-b768-c15b7e92490e admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:16:18.624515 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1299/2589] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:16:18 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:16:18.628508 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b929127e-d37a-43b9-946c-2c669afef6dc admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:16:18.639928 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b929127e-d37a-43b9-946c-2c669afef6dc admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:16:18.645553 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9edaae-a698-4f31-9d2d-005ef962efca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:16:18.646241 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9edaae-a698-4f31-9d2d-005ef962efca admin admin] REQUEST_METHOD: `PUT` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:16:18.646520 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9edaae-a698-4f31-9d2d-005ef962efca admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:16:18.646833 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9edaae-a698-4f31-9d2d-005ef962efca admin admin] PATH_INFO: `/v3/projects/325b8a46dc2a4b63b7d2c8c082abfe56/users/6e03db0c522046bba6b5a7df84f4036b/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:16:18.648764 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9edaae-a698-4f31-9d2d-005ef962efca admin admin] RBAC: Authorizing `identity:create_grant(project_id=325b8a46dc2a4b63b7d2c8c082abfe56, user_id=6e03db0c522046bba6b5a7df84f4036b, role_id=5cf1f4bd2fef46ab9c6eff56d52f9f69)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:16:18.651453 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9edaae-a698-4f31-9d2d-005ef962efca admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:16:18.662823 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1291/2590] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 12:16:18 2022] PUT /identity/v3/projects/325b8a46dc2a4b63b7d2c8c082abfe56/users/6e03db0c522046bba6b5a7df84f4036b/roles/5cf1f4bd2fef46ab9c6eff56d52f9f69 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:16:18.674676 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0a847b-6870-4c4a-8912-c27ea97e8039 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:16:18.674957 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0a847b-6870-4c4a-8912-c27ea97e8039 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:16:18.675232 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0a847b-6870-4c4a-8912-c27ea97e8039 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:16:18.706532 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-5a0a847b-6870-4c4a-8912-c27ea97e8039 None None] MFA Rules not processed for user `6e03db0c522046bba6b5a7df84f4036b`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:16:18.707822 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-5a0a847b-6870-4c4a-8912-c27ea97e8039 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:16:18.726991 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:16:18.785997 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1300/2591] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:16:18 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:16:18.798580 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9edaae-a698-4f31-9d2d-005ef962efca admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:16:18.800564 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-7b9edaae-a698-4f31-9d2d-005ef962efca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:16:18.862388 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9edaae-a698-4f31-9d2d-005ef962efca admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:16:18.866197 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39d9298-001b-4161-83ca-6fd77b327f3c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b1b651c73664d08accc6edf2f79bace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:16:18.866842 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39d9298-001b-4161-83ca-6fd77b327f3c service cinder] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:16:18.867123 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39d9298-001b-4161-83ca-6fd77b327f3c service cinder] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:16:18.867401 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39d9298-001b-4161-83ca-6fd77b327f3c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:16:18.877168 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39d9298-001b-4161-83ca-6fd77b327f3c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:16:18.878776 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39d9298-001b-4161-83ca-6fd77b327f3c service cinder] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:16:18.891028 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1292/2592] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:16:18 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:16:18.990829 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0a847b-6870-4c4a-8912-c27ea97e8039 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:16:19.001562 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0a847b-6870-4c4a-8912-c27ea97e8039 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:16:19.009828 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273dc6f6-2e0c-441c-9b8d-94d9fda55002 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b1b651c73664d08accc6edf2f79bace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:16:19.010471 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273dc6f6-2e0c-441c-9b8d-94d9fda55002 service cinder] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:16:19.010794 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273dc6f6-2e0c-441c-9b8d-94d9fda55002 service cinder] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:16:19.011072 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273dc6f6-2e0c-441c-9b8d-94d9fda55002 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:16:19.070259 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273dc6f6-2e0c-441c-9b8d-94d9fda55002 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:16:19.071761 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273dc6f6-2e0c-441c-9b8d-94d9fda55002 service cinder] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:16:19.080666 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1301/2593] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:16:18 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:16:21.891436 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39d9298-001b-4161-83ca-6fd77b327f3c service cinder] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:16:21.955166 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39d9298-001b-4161-83ca-6fd77b327f3c service cinder] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:16:21.960067 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e873e7d-9d3e-4f83-a94b-5f90a8684cd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:16:21.960698 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e873e7d-9d3e-4f83-a94b-5f90a8684cd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:16:21.960965 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e873e7d-9d3e-4f83-a94b-5f90a8684cd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:16:21.961235 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e873e7d-9d3e-4f83-a94b-5f90a8684cd2 admin admin] PATH_INFO: `/v3/users/1314daa45047484bb4015043e71efa55` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:16:21.962401 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e873e7d-9d3e-4f83-a94b-5f90a8684cd2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1314daa45047484bb4015043e71efa55)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:16:21.964405 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e873e7d-9d3e-4f83-a94b-5f90a8684cd2 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:16:21.975755 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:16:22.013743 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-5e873e7d-9d3e-4f83-a94b-5f90a8684cd2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1314daa45047484bb4015043e71efa55'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:16:22.026324 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-5e873e7d-9d3e-4f83-a94b-5f90a8684cd2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1314daa45047484bb4015043e71efa55'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:16:22.035542 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-5e873e7d-9d3e-4f83-a94b-5f90a8684cd2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1314daa45047484bb4015043e71efa55'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:16:22.036235 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-5e873e7d-9d3e-4f83-a94b-5f90a8684cd2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1314daa45047484bb4015043e71efa55'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:16:22.043825 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-5e873e7d-9d3e-4f83-a94b-5f90a8684cd2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1314daa45047484bb4015043e71efa55'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:16:22.050000 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1293/2594] 10.222.0.59 () {58 vars in 1224 bytes} [Tue Oct 11 12:16:21 2022] DELETE /identity/v3/users/1314daa45047484bb4015043e71efa55 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:16:22.054849 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bcce97-7f61-47bf-8b63-e4ee968bca58 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:16:22.055342 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bcce97-7f61-47bf-8b63-e4ee968bca58 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:16:22.055480 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bcce97-7f61-47bf-8b63-e4ee968bca58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:16:22.076490 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-79bcce97-7f61-47bf-8b63-e4ee968bca58 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:16:22.077732 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-79bcce97-7f61-47bf-8b63-e4ee968bca58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:16:22.137544 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1302/2595] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:16:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 11 12:16:22.147972 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e873e7d-9d3e-4f83-a94b-5f90a8684cd2 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:16:22.149595 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-5e873e7d-9d3e-4f83-a94b-5f90a8684cd2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:16:22.209924 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e873e7d-9d3e-4f83-a94b-5f90a8684cd2 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:16:22.213715 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05149151-deb8-4be7-9954-276620a85e55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e563fe174db4cc19ae6bf8162cc3d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:16:22.214320 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05149151-deb8-4be7-9954-276620a85e55 service neutron] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:16:22.215283 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05149151-deb8-4be7-9954-276620a85e55 service neutron] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:16:22.215670 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05149151-deb8-4be7-9954-276620a85e55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:16:22.226076 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05149151-deb8-4be7-9954-276620a85e55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:16:22.227746 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05149151-deb8-4be7-9954-276620a85e55 service neutron] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:16:22.238440 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1294/2596] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:16:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:16:22.546410 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bcce97-7f61-47bf-8b63-e4ee968bca58 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:16:22.548077 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-79bcce97-7f61-47bf-8b63-e4ee968bca58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:16:22.560638 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bcce97-7f61-47bf-8b63-e4ee968bca58 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:16:22.566261 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9a5985-e6af-4d11-bf2b-74d053368734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:16:22.566448 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9a5985-e6af-4d11-bf2b-74d053368734 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:16:22.566738 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9a5985-e6af-4d11-bf2b-74d053368734 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:16:22.566981 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9a5985-e6af-4d11-bf2b-74d053368734 admin admin] PATH_INFO: `/v3/projects/8dd4f509f7bc46c6afd05ec88982d75c` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:16:22.568180 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9a5985-e6af-4d11-bf2b-74d053368734 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8dd4f509f7bc46c6afd05ec88982d75c)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:16:22.570268 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9a5985-e6af-4d11-bf2b-74d053368734 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:16:22.619255 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:16:22.620806 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-9f9a5985-e6af-4d11-bf2b-74d053368734 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8dd4f509f7bc46c6afd05ec88982d75c', 'request_id': 'req-9f9a5985-e6af-4d11-bf2b-74d053368734'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:16:22.627687 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-9f9a5985-e6af-4d11-bf2b-74d053368734 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8dd4f509f7bc46c6afd05ec88982d75c', 'request_id': 'req-9f9a5985-e6af-4d11-bf2b-74d053368734'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:16:22.634764 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-9f9a5985-e6af-4d11-bf2b-74d053368734 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8dd4f509f7bc46c6afd05ec88982d75c', 'request_id': 'req-9f9a5985-e6af-4d11-bf2b-74d053368734'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:16:22.648658 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-9f9a5985-e6af-4d11-bf2b-74d053368734 admin admin] The token cache is being invalidate because project 8dd4f509f7bc46c6afd05ec88982d75c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81432) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 12:16:22.648949 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-9f9a5985-e6af-4d11-bf2b-74d053368734 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:16:22.649565 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-9f9a5985-e6af-4d11-bf2b-74d053368734 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:16:22.651227 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1303/2597] 10.222.0.59 () {58 vars in 1233 bytes} [Tue Oct 11 12:16:22 2022] DELETE /identity/v3/projects/8dd4f509f7bc46c6afd05ec88982d75c => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:16:22.655552 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05149151-deb8-4be7-9954-276620a85e55 service neutron] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:16:22.665243 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05149151-deb8-4be7-9954-276620a85e55 service neutron] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:16:22.723211 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059b1415-e4c3-4632-b00e-cde37f323d1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:16:22.723855 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059b1415-e4c3-4632-b00e-cde37f323d1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:16:22.724157 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059b1415-e4c3-4632-b00e-cde37f323d1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:16:22.724573 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059b1415-e4c3-4632-b00e-cde37f323d1d admin admin] PATH_INFO: `/v3/users/6e03db0c522046bba6b5a7df84f4036b` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:16:22.725730 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059b1415-e4c3-4632-b00e-cde37f323d1d admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e03db0c522046bba6b5a7df84f4036b)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:16:22.727687 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059b1415-e4c3-4632-b00e-cde37f323d1d admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:16:22.765927 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:DELETE=15,keystone:SELECT=84,keystone:INSERT=1 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:16:22.780421 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-059b1415-e4c3-4632-b00e-cde37f323d1d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e03db0c522046bba6b5a7df84f4036b'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:16:22.797662 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-059b1415-e4c3-4632-b00e-cde37f323d1d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e03db0c522046bba6b5a7df84f4036b'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:16:22.806398 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-059b1415-e4c3-4632-b00e-cde37f323d1d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e03db0c522046bba6b5a7df84f4036b'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:16:22.807355 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-059b1415-e4c3-4632-b00e-cde37f323d1d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e03db0c522046bba6b5a7df84f4036b'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:16:22.814752 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-059b1415-e4c3-4632-b00e-cde37f323d1d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e03db0c522046bba6b5a7df84f4036b'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:16:22.820792 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1295/2598] 10.222.0.59 () {58 vars in 1224 bytes} [Tue Oct 11 12:16:22 2022] DELETE /identity/v3/users/6e03db0c522046bba6b5a7df84f4036b => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:16:23.237903 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9a5985-e6af-4d11-bf2b-74d053368734 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:16:23.239452 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-9f9a5985-e6af-4d11-bf2b-74d053368734 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:16:23.253138 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9a5985-e6af-4d11-bf2b-74d053368734 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:16:23.313072 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2639cf9f-311c-4007-9cf5-f109bb622f07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:16:23.313715 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2639cf9f-311c-4007-9cf5-f109bb622f07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:16:23.313989 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2639cf9f-311c-4007-9cf5-f109bb622f07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:16:23.314256 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2639cf9f-311c-4007-9cf5-f109bb622f07 admin admin] PATH_INFO: `/v3/projects/325b8a46dc2a4b63b7d2c8c082abfe56` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:16:23.315401 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2639cf9f-311c-4007-9cf5-f109bb622f07 admin admin] RBAC: Authorizing `identity:delete_project(project_id=325b8a46dc2a4b63b7d2c8c082abfe56)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:16:23.317349 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2639cf9f-311c-4007-9cf5-f109bb622f07 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:16:23.366442 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-2639cf9f-311c-4007-9cf5-f109bb622f07 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '325b8a46dc2a4b63b7d2c8c082abfe56', 'request_id': 'req-2639cf9f-311c-4007-9cf5-f109bb622f07'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:16:23.373014 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-2639cf9f-311c-4007-9cf5-f109bb622f07 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '325b8a46dc2a4b63b7d2c8c082abfe56', 'request_id': 'req-2639cf9f-311c-4007-9cf5-f109bb622f07'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:16:23.380508 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-2639cf9f-311c-4007-9cf5-f109bb622f07 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '325b8a46dc2a4b63b7d2c8c082abfe56', 'request_id': 'req-2639cf9f-311c-4007-9cf5-f109bb622f07'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:16:23.394452 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-2639cf9f-311c-4007-9cf5-f109bb622f07 admin admin] The token cache is being invalidate because project 325b8a46dc2a4b63b7d2c8c082abfe56 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81432) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 12:16:23.394780 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-2639cf9f-311c-4007-9cf5-f109bb622f07 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:16:23.395741 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-2639cf9f-311c-4007-9cf5-f109bb622f07 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:16:23.397229 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1304/2599] 10.222.0.59 () {58 vars in 1233 bytes} [Tue Oct 11 12:16:23 2022] DELETE /identity/v3/projects/325b8a46dc2a4b63b7d2c8c082abfe56 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:16:23.403420 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecbf9b0-7d50-4d00-89ea-5fb18e923ec0 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:16:23.403935 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecbf9b0-7d50-4d00-89ea-5fb18e923ec0 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:16:23.404238 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecbf9b0-7d50-4d00-89ea-5fb18e923ec0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:16:23.426714 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-7ecbf9b0-7d50-4d00-89ea-5fb18e923ec0 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:16:23.428039 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-7ecbf9b0-7d50-4d00-89ea-5fb18e923ec0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:16:23.488923 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1296/2600] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:16:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:16:23.496775 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3434f3ec-8765-4e20-8ee1-c5f60fe5c8f6 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:16:23.497058 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3434f3ec-8765-4e20-8ee1-c5f60fe5c8f6 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:16:23.497330 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3434f3ec-8765-4e20-8ee1-c5f60fe5c8f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:16:23.517125 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-3434f3ec-8765-4e20-8ee1-c5f60fe5c8f6 None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:16:23.518328 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-3434f3ec-8765-4e20-8ee1-c5f60fe5c8f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:16:23.526468 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1305/2601] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:16:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:16:23.531218 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecbf9b0-7d50-4d00-89ea-5fb18e923ec0 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:16:23.541599 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecbf9b0-7d50-4d00-89ea-5fb18e923ec0 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:16:23.547261 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08197ed-f982-4e22-9fac-ba48cb83516c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:16:23.548021 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08197ed-f982-4e22-9fac-ba48cb83516c admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:16:23.548306 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08197ed-f982-4e22-9fac-ba48cb83516c admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:16:23.548575 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08197ed-f982-4e22-9fac-ba48cb83516c admin admin] PATH_INFO: `/v3/domains` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:16:23.549227 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08197ed-f982-4e22-9fac-ba48cb83516c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81433) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 11 12:16:23.549531 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08197ed-f982-4e22-9fac-ba48cb83516c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:16:23.551167 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08197ed-f982-4e22-9fac-ba48cb83516c admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:16:23.566749 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1297/2602] 10.222.0.59 () {58 vars in 1153 bytes} [Tue Oct 11 12:16:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 11 12:16:23.568135 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3434f3ec-8765-4e20-8ee1-c5f60fe5c8f6 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:16:23.578043 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3434f3ec-8765-4e20-8ee1-c5f60fe5c8f6 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:16:23.583777 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f08371-a0a2-41af-b125-9dc16226a8df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:16:23.584463 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f08371-a0a2-41af-b125-9dc16226a8df admin admin] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:16:23.584741 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f08371-a0a2-41af-b125-9dc16226a8df admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:16:23.585017 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f08371-a0a2-41af-b125-9dc16226a8df admin admin] PATH_INFO: `/v3/projects` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:16:23.585692 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f08371-a0a2-41af-b125-9dc16226a8df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:16:23.587673 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f08371-a0a2-41af-b125-9dc16226a8df admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:16:23.618570 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1306/2603] 10.222.0.59 () {60 vars in 1153 bytes} [Tue Oct 11 12:16:23 2022] POST /identity/v3/projects => generated 391 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 11 12:16:23.619812 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08197ed-f982-4e22-9fac-ba48cb83516c admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:16:23.630105 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08197ed-f982-4e22-9fac-ba48cb83516c admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:16:23.687220 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b444e97-7240-43b6-9324-8ff8f069eeae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:16:23.687811 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b444e97-7240-43b6-9324-8ff8f069eeae admin admin] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:16:23.688108 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b444e97-7240-43b6-9324-8ff8f069eeae admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:16:23.688368 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b444e97-7240-43b6-9324-8ff8f069eeae admin admin] PATH_INFO: `/v3/users` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:16:23.689000 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b444e97-7240-43b6-9324-8ff8f069eeae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:16:23.690972 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b444e97-7240-43b6-9324-8ff8f069eeae admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:16:23.714621 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1298/2604] 10.222.0.59 () {60 vars in 1144 bytes} [Tue Oct 11 12:16:23 2022] POST /identity/v3/users => generated 296 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 11 12:16:23.718681 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f08371-a0a2-41af-b125-9dc16226a8df admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:16:23.728951 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f08371-a0a2-41af-b125-9dc16226a8df admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:16:23.734045 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ae6e38-1fea-4c76-b6a2-6df7b6341ef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:16:23.734702 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ae6e38-1fea-4c76-b6a2-6df7b6341ef0 admin admin] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:16:23.734984 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ae6e38-1fea-4c76-b6a2-6df7b6341ef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:16:23.735254 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ae6e38-1fea-4c76-b6a2-6df7b6341ef0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:16:23.735896 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ae6e38-1fea-4c76-b6a2-6df7b6341ef0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:16:23.737522 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ae6e38-1fea-4c76-b6a2-6df7b6341ef0 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:16:23.748713 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1307/2605] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:16:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 11 12:16:23.752638 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b444e97-7240-43b6-9324-8ff8f069eeae admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:16:23.763445 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b444e97-7240-43b6-9324-8ff8f069eeae admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:16:23.768322 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8447be-970c-4150-ae1b-5e19d2a48ee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:16:23.769029 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8447be-970c-4150-ae1b-5e19d2a48ee4 admin admin] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:16:23.769377 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8447be-970c-4150-ae1b-5e19d2a48ee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:16:23.769718 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8447be-970c-4150-ae1b-5e19d2a48ee4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:16:23.770809 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8447be-970c-4150-ae1b-5e19d2a48ee4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:16:23.772475 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8447be-970c-4150-ae1b-5e19d2a48ee4 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:16:23.786872 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1299/2606] 10.222.0.59 () {58 vars in 1122 bytes} [Tue Oct 11 12:16:23 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 11 12:16:23.791208 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ae6e38-1fea-4c76-b6a2-6df7b6341ef0 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:16:23.801430 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ae6e38-1fea-4c76-b6a2-6df7b6341ef0 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:16:23.806274 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a66209-2a51-4482-85fd-4d82dc758806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:16:23.806956 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a66209-2a51-4482-85fd-4d82dc758806 admin admin] REQUEST_METHOD: `PUT` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:16:23.807240 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a66209-2a51-4482-85fd-4d82dc758806 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:16:23.807849 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a66209-2a51-4482-85fd-4d82dc758806 admin admin] PATH_INFO: `/v3/projects/e1951a1e4179470d938a725e55f13aaf/users/f5980cd4fb8a4671b99ae3cc751eed42/roles/de2d2bd26fff4748a79990f97a107164` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:16:23.823730 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a66209-2a51-4482-85fd-4d82dc758806 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1951a1e4179470d938a725e55f13aaf, user_id=f5980cd4fb8a4671b99ae3cc751eed42, role_id=de2d2bd26fff4748a79990f97a107164)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:16:23.826734 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a66209-2a51-4482-85fd-4d82dc758806 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:16:23.840690 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1308/2607] 10.222.0.59 () {60 vars in 1483 bytes} [Tue Oct 11 12:16:23 2022] PUT /identity/v3/projects/e1951a1e4179470d938a725e55f13aaf/users/f5980cd4fb8a4671b99ae3cc751eed42/roles/de2d2bd26fff4748a79990f97a107164 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:16:23.850894 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96767498-4fbd-4694-b010-f53e30bae767 None None] REQUEST_METHOD: `POST` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:16:23.851184 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96767498-4fbd-4694-b010-f53e30bae767 None None] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:16:23.851454 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96767498-4fbd-4694-b010-f53e30bae767 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:16:23.880001 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.auth.core [None req-96767498-4fbd-4694-b010-f53e30bae767 None None] MFA Rules not processed for user `f5980cd4fb8a4671b99ae3cc751eed42`. Rule list: `[]` (Enabled: `True`). {{(pid=81433) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:16:23.881319 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-96767498-4fbd-4694-b010-f53e30bae767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:16:23.885370 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:16:23.951924 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1300/2608] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:16:23 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:16:23.963126 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a66209-2a51-4482-85fd-4d82dc758806 admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:16:23.964984 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-14a66209-2a51-4482-85fd-4d82dc758806 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:16:23.970921 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:16:24.029208 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a66209-2a51-4482-85fd-4d82dc758806 admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:16:24.032936 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82baea1a-b3dd-432f-8024-071aa4f5e811 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b1b651c73664d08accc6edf2f79bace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:16:24.033548 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82baea1a-b3dd-432f-8024-071aa4f5e811 service cinder] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:16:24.033826 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82baea1a-b3dd-432f-8024-071aa4f5e811 service cinder] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:16:24.034096 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82baea1a-b3dd-432f-8024-071aa4f5e811 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:16:24.042791 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82baea1a-b3dd-432f-8024-071aa4f5e811 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:16:24.044283 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82baea1a-b3dd-432f-8024-071aa4f5e811 service cinder] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:16:24.053785 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1309/2609] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:16:23 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:16:33.948633 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=25 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:16:34.049116 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=25 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:16:44.746861 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96767498-4fbd-4694-b010-f53e30bae767 None None] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:16:44.813218 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96767498-4fbd-4694-b010-f53e30bae767 None None] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:16:44.820465 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0eded00-9838-4f3d-97cd-d38f300c3b82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:16:44.821119 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eded00-9838-4f3d-97cd-d38f300c3b82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:16:44.821376 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eded00-9838-4f3d-97cd-d38f300c3b82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:16:44.821648 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eded00-9838-4f3d-97cd-d38f300c3b82 admin admin] PATH_INFO: `/v3/users/f5980cd4fb8a4671b99ae3cc751eed42` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:16:44.822824 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0eded00-9838-4f3d-97cd-d38f300c3b82 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5980cd4fb8a4671b99ae3cc751eed42)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:16:44.824793 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0eded00-9838-4f3d-97cd-d38f300c3b82 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:16:44.882982 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-f0eded00-9838-4f3d-97cd-d38f300c3b82 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5980cd4fb8a4671b99ae3cc751eed42'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:16:44.896860 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-f0eded00-9838-4f3d-97cd-d38f300c3b82 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5980cd4fb8a4671b99ae3cc751eed42'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:16:44.905974 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-f0eded00-9838-4f3d-97cd-d38f300c3b82 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5980cd4fb8a4671b99ae3cc751eed42'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:16:44.906814 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-f0eded00-9838-4f3d-97cd-d38f300c3b82 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5980cd4fb8a4671b99ae3cc751eed42'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:16:44.914113 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-f0eded00-9838-4f3d-97cd-d38f300c3b82 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5980cd4fb8a4671b99ae3cc751eed42'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:16:44.920143 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1301/2610] 10.222.0.59 () {58 vars in 1224 bytes} [Tue Oct 11 12:16:44 2022] DELETE /identity/v3/users/f5980cd4fb8a4671b99ae3cc751eed42 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:16:44.924742 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd36d7e-0971-42e3-b95b-d6f78e4111da None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:16:44.925308 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd36d7e-0971-42e3-b95b-d6f78e4111da None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:16:44.925308 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd36d7e-0971-42e3-b95b-d6f78e4111da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:16:44.946239 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-6fd36d7e-0971-42e3-b95b-d6f78e4111da None None] MFA Rules not processed for user `a1613582a9844c5fa354d54653ab8e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:16:44.947489 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-6fd36d7e-0971-42e3-b95b-d6f78e4111da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:16:45.012168 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1310/2611] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:16:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:16:45.025148 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0eded00-9838-4f3d-97cd-d38f300c3b82 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:16:45.026742 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-f0eded00-9838-4f3d-97cd-d38f300c3b82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:16:45.088467 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0eded00-9838-4f3d-97cd-d38f300c3b82 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:16:45.093315 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f647ae4-a955-4323-a179-ba1392265832 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e563fe174db4cc19ae6bf8162cc3d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:16:45.093929 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f647ae4-a955-4323-a179-ba1392265832 service neutron] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:16:45.094208 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f647ae4-a955-4323-a179-ba1392265832 service neutron] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:16:45.094478 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f647ae4-a955-4323-a179-ba1392265832 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:16:45.104624 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f647ae4-a955-4323-a179-ba1392265832 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:16:45.106083 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f647ae4-a955-4323-a179-ba1392265832 service neutron] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:16:45.116099 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1302/2612] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:16:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:16:45.396220 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd36d7e-0971-42e3-b95b-d6f78e4111da None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:16:45.397697 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-6fd36d7e-0971-42e3-b95b-d6f78e4111da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:16:45.410183 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd36d7e-0971-42e3-b95b-d6f78e4111da None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:16:45.415869 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76782a2-0fd2-44ad-8545-948e89cec6e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:16:45.416518 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76782a2-0fd2-44ad-8545-948e89cec6e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:16:45.416793 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76782a2-0fd2-44ad-8545-948e89cec6e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:16:45.417058 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76782a2-0fd2-44ad-8545-948e89cec6e1 admin admin] PATH_INFO: `/v3/projects/e1951a1e4179470d938a725e55f13aaf` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:16:45.418328 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76782a2-0fd2-44ad-8545-948e89cec6e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1951a1e4179470d938a725e55f13aaf)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:16:45.420368 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76782a2-0fd2-44ad-8545-948e89cec6e1 admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:16:45.473415 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-c76782a2-0fd2-44ad-8545-948e89cec6e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1951a1e4179470d938a725e55f13aaf', 'request_id': 'req-c76782a2-0fd2-44ad-8545-948e89cec6e1'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:16:45.478874 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-c76782a2-0fd2-44ad-8545-948e89cec6e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1951a1e4179470d938a725e55f13aaf', 'request_id': 'req-c76782a2-0fd2-44ad-8545-948e89cec6e1'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:16:45.485156 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-c76782a2-0fd2-44ad-8545-948e89cec6e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1951a1e4179470d938a725e55f13aaf', 'request_id': 'req-c76782a2-0fd2-44ad-8545-948e89cec6e1'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:16:45.498567 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-c76782a2-0fd2-44ad-8545-948e89cec6e1 admin admin] The token cache is being invalidate because project e1951a1e4179470d938a725e55f13aaf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81432) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 12:16:45.498957 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-c76782a2-0fd2-44ad-8545-948e89cec6e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:16:45.499607 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-c76782a2-0fd2-44ad-8545-948e89cec6e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:16:45.501177 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1311/2613] 10.222.0.59 () {58 vars in 1233 bytes} [Tue Oct 11 12:16:45 2022] DELETE /identity/v3/projects/e1951a1e4179470d938a725e55f13aaf => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:16:55.111760 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:16:55.495875 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:18:13.554583 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f647ae4-a955-4323-a179-ba1392265832 service neutron] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:18:13.556119 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-1f647ae4-a955-4323-a179-ba1392265832 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:18:13.643307 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f647ae4-a955-4323-a179-ba1392265832 service neutron] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:18:13.698187 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d8a565-35ec-4f94-9b2e-4fd201433df9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b1b651c73664d08accc6edf2f79bace', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:18:13.698751 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d8a565-35ec-4f94-9b2e-4fd201433df9 service cinder] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:18:13.699032 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d8a565-35ec-4f94-9b2e-4fd201433df9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:18:13.699281 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d8a565-35ec-4f94-9b2e-4fd201433df9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:18:13.701342 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-70d8a565-35ec-4f94-9b2e-4fd201433df9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:18:13.780416 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d8a565-35ec-4f94-9b2e-4fd201433df9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:18:13.781843 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d8a565-35ec-4f94-9b2e-4fd201433df9 service cinder] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:18:13.791275 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1303/2614] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:18:13 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:18:23.787012 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=87 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:18:23.909011 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489b0173-17e2-415b-84ec-785ef7a66c13 None None] REQUEST_METHOD: `POST` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:18:23.909572 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489b0173-17e2-415b-84ec-785ef7a66c13 None None] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:18:23.909572 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489b0173-17e2-415b-84ec-785ef7a66c13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:18:23.930611 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.auth.core [None req-489b0173-17e2-415b-84ec-785ef7a66c13 None None] MFA Rules not processed for user `82c54e3aa3aa46a28373acb91f32cc9a`. Rule list: `[]` (Enabled: `True`). {{(pid=81432) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 11 12:18:23.931864 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-489b0173-17e2-415b-84ec-785ef7a66c13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:18:23.942942 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1312/2615] 10.222.0.59 () {58 vars in 958 bytes} [Tue Oct 11 12:18:23 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 11 12:18:24.278122 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d8a565-35ec-4f94-9b2e-4fd201433df9 service cinder] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:18:24.280006 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-70d8a565-35ec-4f94-9b2e-4fd201433df9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:18:24.341363 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d8a565-35ec-4f94-9b2e-4fd201433df9 service cinder] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:18:24.345351 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca45e17a-acdb-4567-a74d-8602e2e451b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e563fe174db4cc19ae6bf8162cc3d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:18:24.345956 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca45e17a-acdb-4567-a74d-8602e2e451b7 service neutron] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:18:24.346237 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca45e17a-acdb-4567-a74d-8602e2e451b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:18:24.346510 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca45e17a-acdb-4567-a74d-8602e2e451b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:18:24.348428 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-ca45e17a-acdb-4567-a74d-8602e2e451b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:18:24.358325 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca45e17a-acdb-4567-a74d-8602e2e451b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:18:24.359795 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca45e17a-acdb-4567-a74d-8602e2e451b7 service neutron] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:18:24.367895 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1304/2616] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:18:24 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:18:24.836011 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489b0173-17e2-415b-84ec-785ef7a66c13 None None] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:18:24.837598 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-489b0173-17e2-415b-84ec-785ef7a66c13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:18:24.934818 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489b0173-17e2-415b-84ec-785ef7a66c13 None None] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:18:24.941221 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ace944a-7518-4749-b740-93a26d1848f5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f272cdd1bcc04b4e8261ed533cfd4fd7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:18:24.941864 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ace944a-7518-4749-b740-93a26d1848f5 service placement] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:18:24.942138 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ace944a-7518-4749-b740-93a26d1848f5 service placement] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:18:24.942405 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ace944a-7518-4749-b740-93a26d1848f5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:18:24.944397 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-2ace944a-7518-4749-b740-93a26d1848f5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:18:24.956767 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ace944a-7518-4749-b740-93a26d1848f5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:18:24.958361 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ace944a-7518-4749-b740-93a26d1848f5 service placement] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:18:24.968536 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1313/2617] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:18:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:18:25.469196 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca45e17a-acdb-4567-a74d-8602e2e451b7 service neutron] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:18:25.470919 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-ca45e17a-acdb-4567-a74d-8602e2e451b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:18:25.486738 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca45e17a-acdb-4567-a74d-8602e2e451b7 service neutron] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:18:25.492742 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f888f10e-cbff-4093-b814-f2372bbec8e7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f272cdd1bcc04b4e8261ed533cfd4fd7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:18:25.493339 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f888f10e-cbff-4093-b814-f2372bbec8e7 service placement] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:18:25.493613 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f888f10e-cbff-4093-b814-f2372bbec8e7 service placement] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:18:25.493878 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f888f10e-cbff-4093-b814-f2372bbec8e7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:18:25.495908 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-f888f10e-cbff-4093-b814-f2372bbec8e7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:18:25.507509 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f888f10e-cbff-4093-b814-f2372bbec8e7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:18:25.509071 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f888f10e-cbff-4093-b814-f2372bbec8e7 service placement] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:18:25.519152 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1305/2618] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:18:25 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:18:34.964213 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=47 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:18:35.514927 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=32 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:18:38.980223 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ace944a-7518-4749-b740-93a26d1848f5 service placement] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:18:38.982797 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-2ace944a-7518-4749-b740-93a26d1848f5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:18:39.088446 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ace944a-7518-4749-b740-93a26d1848f5 service placement] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:18:39.093994 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681d03e0-c5f2-4690-8ff8-bf93ee4ace24 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bad5e1cb306b46818096940acc3810b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:18:39.094629 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681d03e0-c5f2-4690-8ff8-bf93ee4ace24 service nova] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:18:39.094908 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681d03e0-c5f2-4690-8ff8-bf93ee4ace24 service nova] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:18:39.095178 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681d03e0-c5f2-4690-8ff8-bf93ee4ace24 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:18:39.096985 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-681d03e0-c5f2-4690-8ff8-bf93ee4ace24 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:18:39.108853 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681d03e0-c5f2-4690-8ff8-bf93ee4ace24 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:18:39.110439 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681d03e0-c5f2-4690-8ff8-bf93ee4ace24 service nova] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:18:39.120628 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1314/2619] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:18:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:18:42.858977 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f888f10e-cbff-4093-b814-f2372bbec8e7 service placement] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:18:42.868689 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f888f10e-cbff-4093-b814-f2372bbec8e7 service placement] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:18:42.873154 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80098489-62c4-4d56-bb61-c4d06754f17e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e563fe174db4cc19ae6bf8162cc3d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:18:42.873715 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80098489-62c4-4d56-bb61-c4d06754f17e service neutron] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:18:42.874025 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80098489-62c4-4d56-bb61-c4d06754f17e service neutron] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:18:42.874292 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80098489-62c4-4d56-bb61-c4d06754f17e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:18:42.876289 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-80098489-62c4-4d56-bb61-c4d06754f17e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:18:42.940438 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80098489-62c4-4d56-bb61-c4d06754f17e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:18:42.941873 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80098489-62c4-4d56-bb61-c4d06754f17e service neutron] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:18:42.952218 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1306/2620] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:18:42 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:18:49.116620 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=46 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:18:52.947681 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=30 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:20:35.869989 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681d03e0-c5f2-4690-8ff8-bf93ee4ace24 service nova] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:20:35.890494 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681d03e0-c5f2-4690-8ff8-bf93ee4ace24 service nova] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:20:35.894583 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67596977-f246-4817-9af9-10d6653225d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e563fe174db4cc19ae6bf8162cc3d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:20:35.895205 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67596977-f246-4817-9af9-10d6653225d4 service neutron] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:20:35.895483 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67596977-f246-4817-9af9-10d6653225d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:20:35.895760 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67596977-f246-4817-9af9-10d6653225d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:20:35.897615 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-67596977-f246-4817-9af9-10d6653225d4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:20:35.911936 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67596977-f246-4817-9af9-10d6653225d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:20:35.913416 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67596977-f246-4817-9af9-10d6653225d4 service neutron] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:20:35.924871 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1315/2621] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:20:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:20:37.105976 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80098489-62c4-4d56-bb61-c4d06754f17e service neutron] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:20:37.107688 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-80098489-62c4-4d56-bb61-c4d06754f17e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:20:37.146460 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80098489-62c4-4d56-bb61-c4d06754f17e service neutron] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:20:37.150482 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd82b61c-1d97-4fee-9733-f2a247c78431 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e563fe174db4cc19ae6bf8162cc3d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:20:37.151203 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd82b61c-1d97-4fee-9733-f2a247c78431 service neutron] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:20:37.151493 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd82b61c-1d97-4fee-9733-f2a247c78431 service neutron] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:20:37.151767 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd82b61c-1d97-4fee-9733-f2a247c78431 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:20:37.153816 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-dd82b61c-1d97-4fee-9733-f2a247c78431 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:20:37.168561 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd82b61c-1d97-4fee-9733-f2a247c78431 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:20:37.169995 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd82b61c-1d97-4fee-9733-f2a247c78431 service neutron] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:20:37.182129 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1307/2622] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:20:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:20:45.920109 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=10 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:20:47.176642 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=15 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:20:48.525314 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67596977-f246-4817-9af9-10d6653225d4 service neutron] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:20:48.534202 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67596977-f246-4817-9af9-10d6653225d4 service neutron] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:20:48.537904 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5418128d-3cfb-44e0-af24-7bd9eac8127a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e563fe174db4cc19ae6bf8162cc3d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:20:48.538508 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5418128d-3cfb-44e0-af24-7bd9eac8127a service neutron] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:20:48.538831 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5418128d-3cfb-44e0-af24-7bd9eac8127a service neutron] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:20:48.539105 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5418128d-3cfb-44e0-af24-7bd9eac8127a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:20:48.540861 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-5418128d-3cfb-44e0-af24-7bd9eac8127a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:20:48.628622 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5418128d-3cfb-44e0-af24-7bd9eac8127a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:20:48.630085 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5418128d-3cfb-44e0-af24-7bd9eac8127a service neutron] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:20:48.650839 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1316/2623] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:20:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:20:51.738115 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd82b61c-1d97-4fee-9733-f2a247c78431 service neutron] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:20:51.755458 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd82b61c-1d97-4fee-9733-f2a247c78431 service neutron] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:20:51.765207 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b834b0-e3e7-4c36-896c-fdecfef75ceb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f272cdd1bcc04b4e8261ed533cfd4fd7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:20:51.766151 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b834b0-e3e7-4c36-896c-fdecfef75ceb service placement] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:20:51.766437 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b834b0-e3e7-4c36-896c-fdecfef75ceb service placement] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:20:51.766844 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b834b0-e3e7-4c36-896c-fdecfef75ceb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:20:51.768824 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-55b834b0-e3e7-4c36-896c-fdecfef75ceb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:20:51.782028 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b834b0-e3e7-4c36-896c-fdecfef75ceb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:20:51.787753 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b834b0-e3e7-4c36-896c-fdecfef75ceb service placement] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:20:51.799429 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1308/2624] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:20:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:20:54.899704 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5418128d-3cfb-44e0-af24-7bd9eac8127a service neutron] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:20:54.911483 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5418128d-3cfb-44e0-af24-7bd9eac8127a service neutron] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:20:54.916869 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ce1465-ad9d-4add-8bf7-83e7fa1c5090 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f272cdd1bcc04b4e8261ed533cfd4fd7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:20:54.917381 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ce1465-ad9d-4add-8bf7-83e7fa1c5090 service placement] REQUEST_METHOD: `GET` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:20:54.917662 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ce1465-ad9d-4add-8bf7-83e7fa1c5090 service placement] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:20:54.917932 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ce1465-ad9d-4add-8bf7-83e7fa1c5090 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:20:54.919673 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-d9ce1465-ad9d-4add-8bf7-83e7fa1c5090 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:20:54.936322 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ce1465-ad9d-4add-8bf7-83e7fa1c5090 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:20:54.938499 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ce1465-ad9d-4add-8bf7-83e7fa1c5090 service placement] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:20:54.950940 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1317/2625] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:20:54 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:20:54.963806 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b834b0-e3e7-4c36-896c-fdecfef75ceb service placement] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:20:54.965349 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-55b834b0-e3e7-4c36-896c-fdecfef75ceb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:20:54.980584 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b834b0-e3e7-4c36-896c-fdecfef75ceb service placement] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:20:54.991090 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6eda67-4868-4d08-b463-ec8d3eadd2b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:20:54.991885 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6eda67-4868-4d08-b463-ec8d3eadd2b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:20:54.992240 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6eda67-4868-4d08-b463-ec8d3eadd2b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:20:54.992687 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6eda67-4868-4d08-b463-ec8d3eadd2b7 admin admin] PATH_INFO: `/v3/users/82c54e3aa3aa46a28373acb91f32cc9a` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:20:54.993927 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6eda67-4868-4d08-b463-ec8d3eadd2b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=82c54e3aa3aa46a28373acb91f32cc9a)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:20:54.995956 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6eda67-4868-4d08-b463-ec8d3eadd2b7 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:20:55.046109 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-3a6eda67-4868-4d08-b463-ec8d3eadd2b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82c54e3aa3aa46a28373acb91f32cc9a'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:20:55.059312 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-3a6eda67-4868-4d08-b463-ec8d3eadd2b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82c54e3aa3aa46a28373acb91f32cc9a'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:20:55.068246 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-3a6eda67-4868-4d08-b463-ec8d3eadd2b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82c54e3aa3aa46a28373acb91f32cc9a'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:20:55.068799 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-3a6eda67-4868-4d08-b463-ec8d3eadd2b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82c54e3aa3aa46a28373acb91f32cc9a'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:20:55.075897 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-3a6eda67-4868-4d08-b463-ec8d3eadd2b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82c54e3aa3aa46a28373acb91f32cc9a'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:20:55.081801 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1309/2626] 10.222.0.59 () {58 vars in 1224 bytes} [Tue Oct 11 12:20:54 2022] DELETE /identity/v3/users/82c54e3aa3aa46a28373acb91f32cc9a => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:20:55.258451 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ce1465-ad9d-4add-8bf7-83e7fa1c5090 service placement] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:20:55.270017 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ce1465-ad9d-4add-8bf7-83e7fa1c5090 service placement] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:20:55.326402 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98fc32c-15d8-48df-98fd-0bd805c7f72a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:20:55.327068 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98fc32c-15d8-48df-98fd-0bd805c7f72a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:20:55.327350 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98fc32c-15d8-48df-98fd-0bd805c7f72a admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:20:55.327624 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98fc32c-15d8-48df-98fd-0bd805c7f72a admin admin] PATH_INFO: `/v3/projects/4f2eaa1959394ae3a4b4b27d00b917d1` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:20:55.328752 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98fc32c-15d8-48df-98fd-0bd805c7f72a admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f2eaa1959394ae3a4b4b27d00b917d1)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:20:55.330750 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98fc32c-15d8-48df-98fd-0bd805c7f72a admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:20:55.385040 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-f98fc32c-15d8-48df-98fd-0bd805c7f72a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f2eaa1959394ae3a4b4b27d00b917d1', 'request_id': 'req-f98fc32c-15d8-48df-98fd-0bd805c7f72a'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:20:55.388841 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:20:55.391368 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-f98fc32c-15d8-48df-98fd-0bd805c7f72a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f2eaa1959394ae3a4b4b27d00b917d1', 'request_id': 'req-f98fc32c-15d8-48df-98fd-0bd805c7f72a'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:20:55.399736 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-f98fc32c-15d8-48df-98fd-0bd805c7f72a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f2eaa1959394ae3a4b4b27d00b917d1', 'request_id': 'req-f98fc32c-15d8-48df-98fd-0bd805c7f72a'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:20:55.414639 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-f98fc32c-15d8-48df-98fd-0bd805c7f72a admin admin] The token cache is being invalidate because project 4f2eaa1959394ae3a4b4b27d00b917d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81432) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 12:20:55.414925 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-f98fc32c-15d8-48df-98fd-0bd805c7f72a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:20:55.415551 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-f98fc32c-15d8-48df-98fd-0bd805c7f72a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:20:55.417037 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1318/2627] 10.222.0.59 () {58 vars in 1233 bytes} [Tue Oct 11 12:20:55 2022] DELETE /identity/v3/projects/4f2eaa1959394ae3a4b4b27d00b917d1 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:20:55.421279 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6eda67-4868-4d08-b463-ec8d3eadd2b7 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:20:55.422925 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-3a6eda67-4868-4d08-b463-ec8d3eadd2b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:20:55.436811 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6eda67-4868-4d08-b463-ec8d3eadd2b7 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:20:55.495764 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f37b594-9b26-452e-88b9-41c84864fa62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:20:55.496388 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f37b594-9b26-452e-88b9-41c84864fa62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:20:55.496666 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f37b594-9b26-452e-88b9-41c84864fa62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:20:55.496955 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f37b594-9b26-452e-88b9-41c84864fa62 admin admin] PATH_INFO: `/v3/users/f09e4d3c403d40ffad9ca134fa802157` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:20:55.498142 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f37b594-9b26-452e-88b9-41c84864fa62 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f09e4d3c403d40ffad9ca134fa802157)` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:20:55.500079 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f37b594-9b26-452e-88b9-41c84864fa62 admin admin] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:20:55.549757 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-1f37b594-9b26-452e-88b9-41c84864fa62 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f09e4d3c403d40ffad9ca134fa802157'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:20:55.560788 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:20:55.563415 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-1f37b594-9b26-452e-88b9-41c84864fa62 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f09e4d3c403d40ffad9ca134fa802157'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:20:55.573217 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-1f37b594-9b26-452e-88b9-41c84864fa62 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f09e4d3c403d40ffad9ca134fa802157'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:20:55.574254 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-1f37b594-9b26-452e-88b9-41c84864fa62 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f09e4d3c403d40ffad9ca134fa802157'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:20:55.581136 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.notifications [None req-1f37b594-9b26-452e-88b9-41c84864fa62 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f09e4d3c403d40ffad9ca134fa802157'} {{(pid=81433) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:20:55.587040 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1310/2628] 10.222.0.59 () {58 vars in 1224 bytes} [Tue Oct 11 12:20:55 2022] DELETE /identity/v3/users/f09e4d3c403d40ffad9ca134fa802157 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:20:55.770820 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98fc32c-15d8-48df-98fd-0bd805c7f72a admin admin] Authenticating user token {{(pid=81432) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:20:55.772447 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.fernet_utils [None req-f98fc32c-15d8-48df-98fd-0bd805c7f72a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81432) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:20:55.785946 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98fc32c-15d8-48df-98fd-0bd805c7f72a admin admin] Validating token access rules against request {{(pid=81432) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:20:55.843204 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db03b9f-b152-4b13-9208-b058e56ba1ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1613582a9844c5fa354d54653ab8e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec9b5254352a410fb1f7cca5a5393294', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81432) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:20:55.843760 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db03b9f-b152-4b13-9208-b058e56ba1ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:20:55.844035 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db03b9f-b152-4b13-9208-b058e56ba1ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:20:55.844306 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db03b9f-b152-4b13-9208-b058e56ba1ba admin admin] PATH_INFO: `/v3/projects/a0ad01844e424ab28fe744762385d05d` {{(pid=81432) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:20:55.845452 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db03b9f-b152-4b13-9208-b058e56ba1ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0ad01844e424ab28fe744762385d05d)` {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:20:55.847434 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db03b9f-b152-4b13-9208-b058e56ba1ba admin admin] RBAC: Authorization granted {{(pid=81432) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:20:55.904868 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-1db03b9f-b152-4b13-9208-b058e56ba1ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0ad01844e424ab28fe744762385d05d', 'request_id': 'req-1db03b9f-b152-4b13-9208-b058e56ba1ba'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:20:55.911150 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-1db03b9f-b152-4b13-9208-b058e56ba1ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0ad01844e424ab28fe744762385d05d', 'request_id': 'req-1db03b9f-b152-4b13-9208-b058e56ba1ba'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:20:55.918025 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-1db03b9f-b152-4b13-9208-b058e56ba1ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0ad01844e424ab28fe744762385d05d', 'request_id': 'req-1db03b9f-b152-4b13-9208-b058e56ba1ba'} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:20:55.933368 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-1db03b9f-b152-4b13-9208-b058e56ba1ba admin admin] The token cache is being invalidate because project a0ad01844e424ab28fe744762385d05d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81432) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 11 12:20:55.933661 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-1db03b9f-b152-4b13-9208-b058e56ba1ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:20:55.934305 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG keystone.notifications [None req-1db03b9f-b152-4b13-9208-b058e56ba1ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81432) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 11 12:20:55.935858 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: [pid: 81432|app: 0|req: 1319/2629] 10.222.0.59 () {58 vars in 1233 bytes} [Tue Oct 11 12:20:55 2022] DELETE /identity/v3/projects/a0ad01844e424ab28fe744762385d05d => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 11 12:21:05.584049 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:DELETE=2,keystone:SELECT=7,keystone:INSERT=1 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:21:05.930477 dv-cinder-820994-6-iscsi devstack@keystone.service[81432]: DEBUG dbcounter [-] [81432] Writing DB stats keystone:DELETE=9,keystone:SELECT=56,keystone:INSERT=2 {{(pid=81432) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 11 12:24:19.781664 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f37b594-9b26-452e-88b9-41c84864fa62 admin admin] Authenticating user token {{(pid=81433) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 11 12:24:19.783254 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-1f37b594-9b26-452e-88b9-41c84864fa62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:24:19.863921 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f37b594-9b26-452e-88b9-41c84864fa62 admin admin] Validating token access rules against request {{(pid=81433) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 11 12:24:19.928620 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c563ebc-0629-4dab-afcd-62e461171284 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f272cdd1bcc04b4e8261ed533cfd4fd7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26ecf552cc9a4e5ca1d380509683dc4a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81433) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 11 12:24:19.929210 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c563ebc-0629-4dab-afcd-62e461171284 service placement] REQUEST_METHOD: `GET` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 11 12:24:19.929484 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c563ebc-0629-4dab-afcd-62e461171284 service placement] SCRIPT_NAME: `/identity` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 11 12:24:19.929752 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c563ebc-0629-4dab-afcd-62e461171284 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81433) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 11 12:24:19.931949 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.fernet_utils [None req-1c563ebc-0629-4dab-afcd-62e461171284 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81433) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 11 12:24:19.944625 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c563ebc-0629-4dab-afcd-62e461171284 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 11 12:24:19.946247 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c563ebc-0629-4dab-afcd-62e461171284 service placement] RBAC: Authorization granted {{(pid=81433) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 11 12:24:19.956425 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: [pid: 81433|app: 0|req: 1311/2630] 10.222.0.59 () {60 vars in 1365 bytes} [Tue Oct 11 12:24:19 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 11 12:24:29.952200 dv-cinder-820994-6-iscsi devstack@keystone.service[81433]: DEBUG dbcounter [-] [81433] Writing DB stats keystone:SELECT=58 {{(pid=81433) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}